Jul 14 10:00:14.093768 ip-10-0-141-78 systemd[1]: Starting Kubernetes Kubelet... Jul 14 10:00:15.646561 ip-10-0-141-78 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 10:00:15.646561 ip-10-0-141-78 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 10:00:15.646561 ip-10-0-141-78 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 10:00:15.646561 ip-10-0-141-78 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 10:00:15.646561 ip-10-0-141-78 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 10:00:15.647620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.647460 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 10:00:15.651066 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651050 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.651066 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651066 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651071 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651074 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651077 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651080 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651083 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651086 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651089 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651092 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651095 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651097 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651100 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651103 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651105 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651108 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651111 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651113 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651120 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651123 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651126 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651129 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.651136 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651131 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651134 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651137 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651140 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651143 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651146 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651161 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651164 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651167 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651169 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651172 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651174 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651177 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651180 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651182 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651185 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651187 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651190 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651192 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651195 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.651643 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651197 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651199 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651203 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651205 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651208 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651210 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651213 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651215 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651218 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651221 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651223 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651226 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651228 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651230 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651234 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651238 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651242 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651245 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651248 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.652134 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651251 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651254 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651256 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651259 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651262 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651264 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651266 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651269 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651271 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651279 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651282 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651285 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651287 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651291 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651293 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651296 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651302 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651304 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651307 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.652604 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651310 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651319 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651323 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651325 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651328 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651331 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651334 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651337 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651341 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651344 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651347 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651350 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651353 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651355 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651358 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651361 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651363 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651366 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.651369 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652079 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.653043 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652085 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652088 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652090 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652093 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652096 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652099 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652102 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652105 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652107 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652110 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652113 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652116 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652120 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652124 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652127 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652131 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652135 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652138 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652141 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652145 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.653578 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652163 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652167 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652171 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652175 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652177 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652180 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652183 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652186 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652189 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652191 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652194 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652197 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652199 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652202 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652205 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652207 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652210 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652212 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652215 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.654108 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652217 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652220 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652223 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652225 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652228 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652231 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652233 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652236 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652239 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652242 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652244 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652247 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652250 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652254 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652256 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652259 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652262 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652264 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652267 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652269 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.654832 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652272 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652274 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652277 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652279 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652282 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652284 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652287 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652289 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652292 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652295 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652297 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652299 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652302 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652304 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652307 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652309 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652312 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652315 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652317 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.655339 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652320 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652322 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652325 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652327 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652330 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652332 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652335 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652339 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652342 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652345 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652347 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652350 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652352 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652354 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652357 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652360 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652362 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652365 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652367 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.655908 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652370 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652731 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652735 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652738 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652740 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652743 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652746 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652748 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652751 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652753 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652756 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652759 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652762 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652764 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652767 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652769 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652773 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652775 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652779 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.656387 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652783 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652787 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652790 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652793 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652795 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652798 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652800 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652803 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652805 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652808 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652810 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652812 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652815 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652818 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652820 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652822 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652825 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652827 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652830 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652832 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.656841 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652835 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652837 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652840 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652842 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652845 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652848 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652851 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652853 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652856 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652859 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652861 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652864 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652867 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652870 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652872 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652875 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652879 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652882 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652886 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.657325 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652889 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652891 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652894 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652897 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652900 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652903 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652906 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652909 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652911 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652914 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652916 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652919 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652921 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652924 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652926 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652929 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652931 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652935 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652937 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652940 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.657789 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652943 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652945 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652949 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652951 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652954 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652956 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652959 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652962 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652964 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652967 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652969 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652972 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652974 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652977 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652979 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652982 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652985 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652987 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652989 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652992 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.658264 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652994 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.652997 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653066 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653077 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653086 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653092 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653098 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653103 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653112 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653117 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653120 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653123 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653128 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653131 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653134 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653137 2569 flags.go:64] FLAG: --cgroup-root="" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653140 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653143 2569 flags.go:64] FLAG: --client-ca-file="" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653146 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653162 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653166 2569 flags.go:64] FLAG: --cluster-domain="" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653169 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653172 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653176 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 10:00:15.658747 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653179 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653183 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653186 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653189 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653192 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653195 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653198 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653201 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653204 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653207 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653210 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653213 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653216 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653219 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653222 2569 flags.go:64] FLAG: --enable-server="true" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653225 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653229 2569 flags.go:64] FLAG: --event-burst="100" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653233 2569 flags.go:64] FLAG: --event-qps="50" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653236 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653239 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653242 2569 flags.go:64] FLAG: --eviction-hard="" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653247 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653252 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653255 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653258 2569 flags.go:64] FLAG: --eviction-soft="" Jul 14 10:00:15.659327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653261 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653264 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653267 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653270 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653273 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653276 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653279 2569 flags.go:64] FLAG: --feature-gates="" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653282 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653285 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653288 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653292 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653295 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653298 2569 flags.go:64] FLAG: --help="false" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653301 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-141-78.ec2.internal" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653304 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653307 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653309 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653313 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653316 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653319 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653322 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653324 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653327 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 10:00:15.659932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653330 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653334 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653337 2569 flags.go:64] FLAG: --kube-reserved="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653340 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653342 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653345 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653352 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653354 2569 flags.go:64] FLAG: --lock-file="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653357 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653360 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653363 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653368 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653371 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653374 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653378 2569 flags.go:64] FLAG: --logging-format="text" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653380 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653384 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653387 2569 flags.go:64] FLAG: --manifest-url="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653390 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653394 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653397 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653401 2569 flags.go:64] FLAG: --max-pods="110" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653404 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653408 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653411 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 10:00:15.660507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653414 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653416 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653419 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653422 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653429 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653432 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653435 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653438 2569 flags.go:64] FLAG: --pod-cidr="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653441 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653446 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653449 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653453 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653456 2569 flags.go:64] FLAG: --port="10250" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653460 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653463 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0037e4f6e9d4d60cf" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653466 2569 flags.go:64] FLAG: --qos-reserved="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653469 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653472 2569 flags.go:64] FLAG: --register-node="true" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653475 2569 flags.go:64] FLAG: --register-with-taints="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653478 2569 flags.go:64] FLAG: --registry-burst="10" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653481 2569 flags.go:64] FLAG: --registry-qps="5" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653484 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653487 2569 flags.go:64] FLAG: --reserved-memory="" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653490 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653493 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 10:00:15.661411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653496 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653499 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653501 2569 flags.go:64] FLAG: --runonce="false" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653504 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653507 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653510 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653521 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653524 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653527 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653530 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653533 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653535 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653538 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653541 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653544 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653547 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653551 2569 flags.go:64] FLAG: --system-cgroups="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653554 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653560 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653563 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653567 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653571 2569 flags.go:64] FLAG: --tls-min-version="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653575 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653577 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653580 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 10:00:15.662482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653584 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653587 2569 flags.go:64] FLAG: --v="2" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653591 2569 flags.go:64] FLAG: --version="false" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653594 2569 flags.go:64] FLAG: --vmodule="" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653598 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.653601 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653717 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653720 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653723 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653726 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653729 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653732 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653735 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653737 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653740 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653742 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653745 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653748 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653750 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653753 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653755 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653758 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.663535 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653761 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653765 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653767 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653770 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653773 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653777 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653780 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653782 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653785 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653788 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653790 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653793 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653795 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653798 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653802 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653806 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653809 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653812 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653815 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.664522 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653818 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653821 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653824 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653826 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653829 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653831 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653834 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653837 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653839 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653842 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653844 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653847 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653849 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653852 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653855 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653858 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653861 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653863 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653869 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653872 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.665126 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653875 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653878 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.653922 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654284 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654293 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654297 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654303 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654308 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654312 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654317 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654321 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654326 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654331 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654335 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654341 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654345 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654355 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654360 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654365 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654369 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.665969 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654373 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654380 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654386 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654390 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654395 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654399 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654406 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654411 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654420 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654425 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654431 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654446 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654451 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654455 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654459 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654464 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654469 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654473 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654478 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.666769 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654483 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654492 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654535 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654541 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.654547 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.655410 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.663078 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.663100 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.663128 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.663137 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663240 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663249 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663254 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663259 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.667298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663263 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663267 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663272 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663277 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663282 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663286 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663290 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663295 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663299 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663304 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663308 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663313 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663317 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663321 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663326 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663330 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663335 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663340 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663344 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663349 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.667714 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663353 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663357 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663362 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663366 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663371 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663376 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663380 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663385 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663389 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663393 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663397 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663401 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663405 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663410 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663414 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663418 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663423 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663428 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663432 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663436 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.668253 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663440 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663445 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663449 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663456 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663462 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663467 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663471 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663476 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663481 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663486 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663490 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663495 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663499 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663503 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663507 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663511 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663516 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663521 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663526 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.668726 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663530 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663534 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663538 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663542 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663548 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663554 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663559 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663564 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663569 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663573 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663578 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663582 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663586 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663591 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663595 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663599 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663604 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663610 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663616 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.669371 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663621 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663625 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663629 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663634 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663639 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663643 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663647 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663651 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663655 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663660 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663664 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663669 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663674 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663678 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663682 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663687 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.669895 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663691 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.663699 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663870 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663877 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663883 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663888 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663892 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663897 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663901 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663906 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663910 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663914 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663918 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663922 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663926 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 10:00:15.670305 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663931 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663935 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663939 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663943 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663947 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663952 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663956 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663960 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663964 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663968 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663972 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663976 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663981 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663985 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663990 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663994 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.663998 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664002 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664006 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664011 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 10:00:15.670701 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664015 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664024 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664029 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664033 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664037 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664041 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664046 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664050 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664054 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664059 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664063 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664067 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664071 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664075 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664079 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664083 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664090 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664095 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664100 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 10:00:15.671222 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664105 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664111 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664117 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664122 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664126 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664131 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664136 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664141 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664145 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664165 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664170 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664173 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664177 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664180 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664184 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664191 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664196 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664200 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664203 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 10:00:15.671979 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664207 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664210 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664214 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664218 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664222 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664226 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664231 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664235 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664239 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664243 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664247 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664251 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664256 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664260 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664264 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664268 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664272 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664276 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664281 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 10:00:15.672752 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664285 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664289 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664293 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664297 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664301 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664306 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664310 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664314 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:15.664318 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.664328 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.664494 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.667258 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.668882 2569 server.go:1019] "Starting client certificate rotation" Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.668990 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 10:00:15.673553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.669020 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 10:00:15.673952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.671843 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 10:00:15.673952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.673646 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 10:00:15.685461 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.685445 2569 log.go:25] "Validated CRI v1 runtime API" Jul 14 10:00:15.689125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.689106 2569 log.go:25] "Validated CRI v1 image API" Jul 14 10:00:15.690014 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.689986 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 10:00:15.692836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.692797 2569 fs.go:135] Filesystem UUIDs: map[09cc0254-f6f6-47ef-814a-98bd393c3b16:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dc278581-4a9e-464a-ac9f-32eba60b73e3:/dev/nvme0n1p4] Jul 14 10:00:15.692895 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.692837 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 10:00:15.695594 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.695576 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 10:00:15.699198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.699080 2569 manager.go:216] Machine: {Timestamp:2026-07-14 10:00:15.69791258 +0000 UTC m=+0.331976411 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100412 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b24113149561f3589cb38b53d5c8e SystemUUID:ec2b2411-3149-561f-3589-cb38b53d5c8e BootID:842c99c8-dd53-45ae-b03b-40b2f0be2639 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:d2:96:0f:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:d2:96:0f:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:bf:46:9d:ba:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 10:00:15.699198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.699197 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 10:00:15.699299 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.699278 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 10:00:15.702192 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.702169 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 10:00:15.702330 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.702196 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-78.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 10:00:15.702836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.702826 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 10:00:15.702865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.702838 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 10:00:15.702865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.702845 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 10:00:15.704237 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.704226 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 10:00:15.704981 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.704972 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 10:00:15.707610 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707599 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 14 10:00:15.707732 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707723 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 10:00:15.707766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707752 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 14 10:00:15.707766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707761 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 10:00:15.707821 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707778 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 10:00:15.707821 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707787 2569 kubelet.go:398] "Adding apiserver pod source" Jul 14 10:00:15.707821 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.707795 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 10:00:15.708188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.708174 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 10:00:15.708227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.708196 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 10:00:15.711541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.711515 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 10:00:15.713463 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.713447 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 10:00:15.713507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.713480 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 10:00:15.714108 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714096 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 10:00:15.714160 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714126 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 10:00:15.714160 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714140 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 10:00:15.714228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714165 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 10:00:15.714228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714182 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 10:00:15.714228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714196 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 10:00:15.714228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714214 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 10:00:15.714228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714227 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 10:00:15.714363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714240 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 10:00:15.714363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714250 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 10:00:15.714363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714263 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 10:00:15.714363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.714274 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 10:00:15.715180 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.715169 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 10:00:15.715212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.715181 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 10:00:15.718522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.718505 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-78.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 10:00:15.718602 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.718566 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gkt7j" Jul 14 10:00:15.718653 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.718616 2569 server.go:1268] "Started kubelet" Jul 14 10:00:15.718653 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.718612 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 10:00:15.718767 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.718722 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 10:00:15.718921 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.718901 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 10:00:15.719016 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.718959 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 10:00:15.719102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.719087 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 10:00:15.719625 ip-10-0-141-78 systemd[1]: Started Kubernetes Kubelet. Jul 14 10:00:15.721722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.721648 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 10:00:15.721951 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.721933 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 14 10:00:15.725448 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.725431 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 10:00:15.725689 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.724630 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-78.ec2.internal.18c21fa99bf4e611 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-78.ec2.internal,UID:ip-10-0-141-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-78.ec2.internal,},FirstTimestamp:2026-07-14 10:00:15.718589969 +0000 UTC m=+0.352653803,LastTimestamp:2026-07-14 10:00:15.718589969 +0000 UTC m=+0.352653803,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-78.ec2.internal,}" Jul 14 10:00:15.726611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.726597 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gkt7j" Jul 14 10:00:15.727315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.727296 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 10:00:15.728019 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.727863 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:15.728019 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.727931 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 10:00:15.728290 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728268 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 10:00:15.728394 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728295 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 10:00:15.728394 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728383 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 10:00:15.728496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728458 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 14 10:00:15.728496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728466 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 14 10:00:15.728931 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728720 2569 factory.go:55] Registering systemd factory Jul 14 10:00:15.728931 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.728735 2569 factory.go:223] Registration of the systemd container factory successfully Jul 14 10:00:15.729697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.729674 2569 factory.go:153] Registering CRI-O factory Jul 14 10:00:15.729697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.729693 2569 factory.go:223] Registration of the crio container factory successfully Jul 14 10:00:15.729862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.729821 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 10:00:15.729862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.729857 2569 factory.go:103] Registering Raw factory Jul 14 10:00:15.729973 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.729875 2569 manager.go:1196] Started watching for new ooms in manager Jul 14 10:00:15.730315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.730300 2569 manager.go:349] Starting recovery of all containers Jul 14 10:00:15.731963 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.731939 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 10:00:15.732044 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.732004 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 10:00:15.739554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.739529 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 10:00:15.741781 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.741764 2569 manager.go:354] Recovery completed Jul 14 10:00:15.745887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.745874 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.747673 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.747657 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.747744 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.747685 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.747744 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.747695 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.748229 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748215 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 10:00:15.748229 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748227 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 10:00:15.748345 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748231 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.748345 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748241 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 14 10:00:15.748885 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748870 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.748958 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748896 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.748958 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.748912 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.749339 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.749329 2569 policy_none.go:49] "None policy: Start" Jul 14 10:00:15.749383 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.749344 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 10:00:15.749383 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.749354 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 10:00:15.750270 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.750260 2569 policy_none.go:47] "Start" Jul 14 10:00:15.784573 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.784423 2569 manager.go:337] "Starting Device Plugin manager" Jul 14 10:00:15.784669 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.784611 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 10:00:15.784669 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.784623 2569 server.go:87] "Starting device plugin registration server" Jul 14 10:00:15.784876 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.784863 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 10:00:15.784907 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.784878 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 10:00:15.784984 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.784964 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 10:00:15.785071 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.785065 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 10:00:15.785115 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.785075 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 10:00:15.785628 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.785612 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 10:00:15.785707 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.785657 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:15.864224 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.864195 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 10:00:15.865414 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.864239 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 10:00:15.865414 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.864275 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 10:00:15.865414 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.864316 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 10:00:15.867106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.867087 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:15.885244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.885226 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.886428 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.886413 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.886516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.886446 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.886516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.886460 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.886516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.886503 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-78.ec2.internal" Jul 14 10:00:15.894833 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.894811 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-78.ec2.internal" Jul 14 10:00:15.894833 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.894832 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-78.ec2.internal\": node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:15.910055 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.910035 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:15.964947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.964911 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal"] Jul 14 10:00:15.965047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.964988 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.965988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.965975 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.966060 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.966003 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.966060 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.966018 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.967643 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.967631 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.967792 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.967779 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:15.967826 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.967808 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.968388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968374 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.968455 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968399 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.968455 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968417 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.968562 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968480 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.968562 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968503 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.968562 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.968512 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.970874 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.970287 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:15.970874 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.970329 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 10:00:15.971709 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.971689 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientMemory" Jul 14 10:00:15.971788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.971719 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 10:00:15.971788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:15.971730 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeHasSufficientPID" Jul 14 10:00:15.999923 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:15.999898 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-78.ec2.internal\" not found" node="ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.004298 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.004279 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-78.ec2.internal\" not found" node="ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.010323 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.010309 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.030741 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.030720 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5af106930072935ac7f96a0491100845-config\") pod \"kube-apiserver-proxy-ip-10-0-141-78.ec2.internal\" (UID: \"5af106930072935ac7f96a0491100845\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.030822 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.030744 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.030822 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.030763 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.110785 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.110695 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.131052 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5af106930072935ac7f96a0491100845-config\") pod \"kube-apiserver-proxy-ip-10-0-141-78.ec2.internal\" (UID: \"5af106930072935ac7f96a0491100845\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.131130 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.131130 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131092 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.131227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131130 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.131227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131137 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5af106930072935ac7f96a0491100845-config\") pod \"kube-apiserver-proxy-ip-10-0-141-78.ec2.internal\" (UID: \"5af106930072935ac7f96a0491100845\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.131227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.131146 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01c463973672e11a273c693fb7db70e9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal\" (UID: \"01c463973672e11a273c693fb7db70e9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.211538 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.211496 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.302005 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.301965 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.306870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.306853 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.312510 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.312494 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.413069 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.412975 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.513568 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.513534 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.614282 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.614254 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-78.ec2.internal\" not found" Jul 14 10:00:16.614930 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.614915 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:16.627916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.627894 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.642677 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.642650 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 10:00:16.644731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.644715 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" Jul 14 10:00:16.651510 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.651491 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 10:00:16.668618 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.668562 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 10:00:16.668744 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.668721 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 10:00:16.668779 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.668748 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 10:00:16.708909 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.708874 2569 apiserver.go:52] "Watching apiserver" Jul 14 10:00:16.725596 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.725573 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 10:00:16.727484 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.727461 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-dgbms","openshift-network-diagnostics/network-check-target-f9qfj","openshift-network-operator/iptables-alerter-qnx4d","kube-system/global-pull-secret-syncer-7nmln","kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547","openshift-cluster-node-tuning-operator/tuned-6npvl","openshift-multus/multus-additional-cni-plugins-hqpdc","openshift-multus/multus-r6dc4","openshift-ovn-kubernetes/ovnkube-node-h5zh8","kube-system/konnectivity-agent-tjjsb","openshift-dns/node-resolver-vm4nd","openshift-image-registry/node-ca-khgqj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal"] Jul 14 10:00:16.728513 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.728495 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 10:00:16.729101 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.729082 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.730354 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.730337 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.731454 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731436 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.731642 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731618 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 09:55:15 +0000 UTC" deadline="2028-03-06 01:03:54.239203447 +0000 UTC" Jul 14 10:00:16.731642 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731642 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14415h3m37.507564463s" Jul 14 10:00:16.731760 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731676 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 10:00:16.731760 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731701 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.731951 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.731935 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.732035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732017 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 10:00:16.732396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732381 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 10:00:16.732558 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732543 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n8bmq\"" Jul 14 10:00:16.732647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732598 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 10:00:16.732701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732679 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.732865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.732846 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.733228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.733208 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.733408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.733391 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tzllq\"" Jul 14 10:00:16.734009 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.733981 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.734009 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734013 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.734247 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734227 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fv9tt\"" Jul 14 10:00:16.734693 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734670 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.734780 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734727 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-kubelet\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734780 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734761 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-systemd-units\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-var-lib-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734848 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-config\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734914 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-etc-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.734971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734946 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.735125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734979 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 10:00:16.735232 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.734946 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-ovn\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735232 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735222 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-env-overrides\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735331 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735253 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-script-lib\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735381 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v29ng\" (UniqueName: \"kubernetes.io/projected/784d21c1-2b89-4930-acae-af220fb6fe25-kube-api-access-v29ng\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735381 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-slash\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735392 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-systemd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735446 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735479 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-bin\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735503 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.735643 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735560 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-netns\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735643 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735591 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 10:00:16.735728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735592 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-netd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735707 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735815 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735739 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/784d21c1-2b89-4930-acae-af220fb6fe25-ovn-node-metrics-cert\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735815 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735763 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-node-log\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735791 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-log-socket\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.735902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735861 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 10:00:16.735902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.735878 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lgk6s\"" Jul 14 10:00:16.736080 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.736062 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 10:00:16.736606 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.736309 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.737198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.737138 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.737304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.737263 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qnj4b\"" Jul 14 10:00:16.737982 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.737956 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.738184 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.738161 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.738668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.738650 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 10:00:16.738904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.738854 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.738989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.738921 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.739102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.739089 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6scfq\"" Jul 14 10:00:16.739468 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.739450 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.740383 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.740363 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 10:00:16.740573 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.740562 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f76jh\"" Jul 14 10:00:16.740908 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.740889 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:16.740998 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.740975 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:16.741064 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.741048 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 10:00:16.741411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.741395 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 10:00:16.741655 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.741641 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbtjm\"" Jul 14 10:00:16.741714 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.741687 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 10:00:16.741878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.741866 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 10:00:16.742185 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.742159 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.742245 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.742229 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:16.743672 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.743656 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.744912 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.744897 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.744974 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.744951 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:16.746071 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.746057 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-29xmc\"" Jul 14 10:00:16.746117 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.746076 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 10:00:16.746174 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.746059 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 10:00:16.763007 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.762990 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-47xj6" Jul 14 10:00:16.773335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.773316 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-47xj6" Jul 14 10:00:16.830438 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.830414 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 10:00:16.836467 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836447 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-ovn\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.836582 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836483 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71976dd2-2c7d-406a-8e1a-ffc873e42174-tmp-dir\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.836582 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-sys\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.836582 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836529 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-registration-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.836582 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836554 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ee471c5-e56c-45c5-b717-c690e2d36229-host\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.836582 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-os-release\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-slash\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836553 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-ovn\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836622 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-modprobe-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836641 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-slash\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836648 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rzzb\" (UniqueName: \"kubernetes.io/projected/37855e49-213f-467c-8c14-40ed6438305a-kube-api-access-5rzzb\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836683 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nzwl\" (UniqueName: \"kubernetes.io/projected/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kube-api-access-7nzwl\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836724 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-kubelet-config\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836744 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.836793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836775 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-daemon-config\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-netns\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-etc-tuned\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836884 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-netns\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-bin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836926 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-hostroot\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836940 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-multus-certs\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836960 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-log-socket\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836975 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-config\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-env-overrides\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.836992 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-log-socket\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pqpn\" (UniqueName: \"kubernetes.io/projected/71976dd2-2c7d-406a-8e1a-ffc873e42174-kube-api-access-2pqpn\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837021 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-run\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-host\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837053 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7tgf\" (UniqueName: \"kubernetes.io/projected/4ee471c5-e56c-45c5-b717-c690e2d36229-kube-api-access-w7tgf\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837083 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/888bf1ef-3b9d-4eb9-b4cc-795798943c72-iptables-alerter-script\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7v4q\" (UniqueName: \"kubernetes.io/projected/888bf1ef-3b9d-4eb9-b4cc-795798943c72-kube-api-access-r7v4q\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.837127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837124 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-conf-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysconfig\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837195 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-conf\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837220 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-socket-dir-parent\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837271 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-netd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837307 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837311 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-netd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837331 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-kubernetes\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837351 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837355 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-systemd\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-socket-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837410 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-system-cni-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837427 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837442 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-etc-kubernetes\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-kubelet\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.837769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837492 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-systemd-units\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837518 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v29ng\" (UniqueName: \"kubernetes.io/projected/784d21c1-2b89-4930-acae-af220fb6fe25-kube-api-access-v29ng\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837543 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-kubelet\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837561 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trfph\" (UniqueName: \"kubernetes.io/projected/d27d3101-05df-4486-a7e4-1dcb0fc993bc-kube-api-access-trfph\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837566 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-systemd-units\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837592 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837610 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-etc-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837635 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-lib-modules\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837655 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-sys-fs\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837672 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-etc-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888bf1ef-3b9d-4eb9-b4cc-795798943c72-host-slash\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837712 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-cnibin\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837739 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837761 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-bin\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837782 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837785 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cnibin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-k8s-cni-cncf-io\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837823 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-host-cni-bin\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837853 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-kubelet\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837871 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7773ae2-7435-415e-8bbf-7fa42bdb462e-agent-certs\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837909 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-device-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837933 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-os-release\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837970 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-tmp\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837983 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.837985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838017 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cni-binary-copy\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brqk9\" (UniqueName: \"kubernetes.io/projected/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-kube-api-access-brqk9\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838049 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-multus\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-systemd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838078 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71976dd2-2c7d-406a-8e1a-ffc873e42174-hosts-file\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838110 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-run-systemd\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838128 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-var-lib-kubelet\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.838843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838143 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838178 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838197 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btl5b\" (UniqueName: \"kubernetes.io/projected/39181225-4694-44bb-9536-ea89b9880b1a-kube-api-access-btl5b\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838215 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/784d21c1-2b89-4930-acae-af220fb6fe25-ovn-node-metrics-cert\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838230 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7773ae2-7435-415e-8bbf-7fa42bdb462e-konnectivity-ca\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838265 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838279 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-system-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838293 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-netns\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838315 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-node-log\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838341 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-var-lib-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838362 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-node-log\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838389 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/784d21c1-2b89-4930-acae-af220fb6fe25-var-lib-openvswitch\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838418 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-script-lib\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838488 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ee471c5-e56c-45c5-b717-c690e2d36229-serviceca\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838508 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-dbus\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.839332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.838632 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 10:00:16.840023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.839974 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-env-overrides\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.840269 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.840246 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-config\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.840597 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.840575 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/784d21c1-2b89-4930-acae-af220fb6fe25-ovnkube-script-lib\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.841683 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.841667 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/784d21c1-2b89-4930-acae-af220fb6fe25-ovn-node-metrics-cert\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.846263 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.846243 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v29ng\" (UniqueName: \"kubernetes.io/projected/784d21c1-2b89-4930-acae-af220fb6fe25-kube-api-access-v29ng\") pod \"ovnkube-node-h5zh8\" (UID: \"784d21c1-2b89-4930-acae-af220fb6fe25\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:16.896401 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.896375 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:16.939376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939355 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-kubelet\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.939496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7773ae2-7435-415e-8bbf-7fa42bdb462e-agent-certs\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.939496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-device-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.939496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939474 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-os-release\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939489 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-kubelet\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939498 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-tmp\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939539 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-device-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939552 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939582 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cni-binary-copy\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939607 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brqk9\" (UniqueName: \"kubernetes.io/projected/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-kube-api-access-brqk9\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.939647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939632 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-multus\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.940366 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.939618 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-os-release\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.940366 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940191 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-multus\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.940366 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71976dd2-2c7d-406a-8e1a-ffc873e42174-hosts-file\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.940591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940466 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-var-lib-kubelet\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.940591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940499 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/71976dd2-2c7d-406a-8e1a-ffc873e42174-hosts-file\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.940591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940571 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-var-lib-kubelet\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.940727 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940580 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cni-binary-copy\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.940727 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940619 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.940727 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940673 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.940727 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940705 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btl5b\" (UniqueName: \"kubernetes.io/projected/39181225-4694-44bb-9536-ea89b9880b1a-kube-api-access-btl5b\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.940727 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940714 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940797 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7773ae2-7435-415e-8bbf-7fa42bdb462e-konnectivity-ca\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940808 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940838 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940869 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940895 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-system-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940917 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-netns\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940951 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.941004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.940983 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ee471c5-e56c-45c5-b717-c690e2d36229-serviceca\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941015 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-dbus\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941066 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71976dd2-2c7d-406a-8e1a-ffc873e42174-tmp-dir\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-sys\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941121 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-registration-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ee471c5-e56c-45c5-b717-c690e2d36229-host\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941199 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-os-release\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941199 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-etc-selinux\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941238 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-modprobe-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941276 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rzzb\" (UniqueName: \"kubernetes.io/projected/37855e49-213f-467c-8c14-40ed6438305a-kube-api-access-5rzzb\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nzwl\" (UniqueName: \"kubernetes.io/projected/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kube-api-access-7nzwl\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941347 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-kubelet-config\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941370 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f7773ae2-7435-415e-8bbf-7fa42bdb462e-konnectivity-ca\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.941380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941378 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941090 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-daemon-config\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-etc-tuned\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941457 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-registration-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941474 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-sys\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941279 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-system-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941534 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ee471c5-e56c-45c5-b717-c690e2d36229-host\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941565 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-netns\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941595 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71976dd2-2c7d-406a-8e1a-ffc873e42174-tmp-dir\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-bin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-hostroot\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941683 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-var-lib-cni-bin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.941734 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941795 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-os-release\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.941918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.941808 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-modprobe-d\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.941941 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:17.441920477 +0000 UTC m=+2.075984313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-multus-certs\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942133 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942162 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-multus-certs\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942169 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-hostroot\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942247 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942462 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-dbus\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942507 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pqpn\" (UniqueName: \"kubernetes.io/projected/71976dd2-2c7d-406a-8e1a-ffc873e42174-kube-api-access-2pqpn\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.942563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-run\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942569 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-host\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942583 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ee471c5-e56c-45c5-b717-c690e2d36229-serviceca\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942602 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7tgf\" (UniqueName: \"kubernetes.io/projected/4ee471c5-e56c-45c5-b717-c690e2d36229-kube-api-access-w7tgf\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942651 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/888bf1ef-3b9d-4eb9-b4cc-795798943c72-iptables-alerter-script\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942680 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7v4q\" (UniqueName: \"kubernetes.io/projected/888bf1ef-3b9d-4eb9-b4cc-795798943c72-kube-api-access-r7v4q\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942711 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-conf-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942740 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysconfig\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942776 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-conf\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942775 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-daemon-config\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942806 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942856 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-conf-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942887 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-host\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-socket-dir-parent\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942946 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-kubernetes\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942982 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-systemd\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-socket-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.945620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943050 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-system-cni-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943081 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-etc-kubernetes\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943122 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysconfig\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943166 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.942949 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-socket-dir-parent\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943255 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-systemd\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943274 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f7773ae2-7435-415e-8bbf-7fa42bdb462e-agent-certs\") pod \"konnectivity-agent-tjjsb\" (UID: \"f7773ae2-7435-415e-8bbf-7fa42bdb462e\") " pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943303 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-sysctl-conf\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943327 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-system-cni-dir\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943384 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-socket-dir\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943390 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-multus-cni-dir\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943447 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-etc-kubernetes\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.943490 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943494 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-run\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943546 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-kubelet-config\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.943557 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:17.443532947 +0000 UTC m=+2.077596783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:16.946470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943488 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/888bf1ef-3b9d-4eb9-b4cc-795798943c72-iptables-alerter-script\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943584 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trfph\" (UniqueName: \"kubernetes.io/projected/d27d3101-05df-4486-a7e4-1dcb0fc993bc-kube-api-access-trfph\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943620 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943626 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-etc-kubernetes\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943653 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-lib-modules\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943817 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/39181225-4694-44bb-9536-ea89b9880b1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-sys-fs\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943907 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888bf1ef-3b9d-4eb9-b4cc-795798943c72-host-slash\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943937 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-cnibin\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943962 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37855e49-213f-467c-8c14-40ed6438305a-lib-modules\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.943988 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cnibin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/888bf1ef-3b9d-4eb9-b4cc-795798943c72-host-slash\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944023 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-k8s-cni-cncf-io\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944073 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f3f6e5a-125e-47cb-805a-bd114123f2bc-sys-fs\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944136 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-host-run-k8s-cni-cncf-io\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944170 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d27d3101-05df-4486-a7e4-1dcb0fc993bc-cnibin\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.944202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39181225-4694-44bb-9536-ea89b9880b1a-cnibin\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.947281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.945521 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-tmp\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.947795 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.947330 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37855e49-213f-467c-8c14-40ed6438305a-etc-tuned\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.950249 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.950229 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:16.950346 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.950256 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:16.950346 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.950271 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:16.950346 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:16.950324 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:17.450307164 +0000 UTC m=+2.084371000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:16.952289 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.952236 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7tgf\" (UniqueName: \"kubernetes.io/projected/4ee471c5-e56c-45c5-b717-c690e2d36229-kube-api-access-w7tgf\") pod \"node-ca-khgqj\" (UID: \"4ee471c5-e56c-45c5-b717-c690e2d36229\") " pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:16.952403 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.952381 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pqpn\" (UniqueName: \"kubernetes.io/projected/71976dd2-2c7d-406a-8e1a-ffc873e42174-kube-api-access-2pqpn\") pod \"node-resolver-vm4nd\" (UID: \"71976dd2-2c7d-406a-8e1a-ffc873e42174\") " pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:16.952462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.952414 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brqk9\" (UniqueName: \"kubernetes.io/projected/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-kube-api-access-brqk9\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:16.952462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.952419 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nzwl\" (UniqueName: \"kubernetes.io/projected/9f3f6e5a-125e-47cb-805a-bd114123f2bc-kube-api-access-7nzwl\") pod \"aws-ebs-csi-driver-node-2q547\" (UID: \"9f3f6e5a-125e-47cb-805a-bd114123f2bc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:16.952559 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.952518 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trfph\" (UniqueName: \"kubernetes.io/projected/d27d3101-05df-4486-a7e4-1dcb0fc993bc-kube-api-access-trfph\") pod \"multus-r6dc4\" (UID: \"d27d3101-05df-4486-a7e4-1dcb0fc993bc\") " pod="openshift-multus/multus-r6dc4" Jul 14 10:00:16.952693 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:16.952673 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c463973672e11a273c693fb7db70e9.slice/crio-698cc2ad9f64ff37b83d686551c15e70a22e13f3298add87a8cda576d9fc239c WatchSource:0}: Error finding container 698cc2ad9f64ff37b83d686551c15e70a22e13f3298add87a8cda576d9fc239c: Status 404 returned error can't find the container with id 698cc2ad9f64ff37b83d686551c15e70a22e13f3298add87a8cda576d9fc239c Jul 14 10:00:16.953324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.953306 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7v4q\" (UniqueName: \"kubernetes.io/projected/888bf1ef-3b9d-4eb9-b4cc-795798943c72-kube-api-access-r7v4q\") pod \"iptables-alerter-qnx4d\" (UID: \"888bf1ef-3b9d-4eb9-b4cc-795798943c72\") " pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:16.953412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.953341 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rzzb\" (UniqueName: \"kubernetes.io/projected/37855e49-213f-467c-8c14-40ed6438305a-kube-api-access-5rzzb\") pod \"tuned-6npvl\" (UID: \"37855e49-213f-467c-8c14-40ed6438305a\") " pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:16.953468 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.953449 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btl5b\" (UniqueName: \"kubernetes.io/projected/39181225-4694-44bb-9536-ea89b9880b1a-kube-api-access-btl5b\") pod \"multus-additional-cni-plugins-hqpdc\" (UID: \"39181225-4694-44bb-9536-ea89b9880b1a\") " pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:16.957998 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:16.957985 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 10:00:17.059458 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.059423 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:17.069667 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.069626 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784d21c1_2b89_4930_acae_af220fb6fe25.slice/crio-a2392c717318c4fd2e490acb63a2f320f3fcbebc29ad2aa5faf7971e78630bb1 WatchSource:0}: Error finding container a2392c717318c4fd2e490acb63a2f320f3fcbebc29ad2aa5faf7971e78630bb1: Status 404 returned error can't find the container with id a2392c717318c4fd2e490acb63a2f320f3fcbebc29ad2aa5faf7971e78630bb1 Jul 14 10:00:17.081009 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.080992 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vm4nd" Jul 14 10:00:17.087930 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.087908 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71976dd2_2c7d_406a_8e1a_ffc873e42174.slice/crio-bb0055b9b61dd03e030cde46ad02f37e072d395f72b57e4e12c52c0ab1605c60 WatchSource:0}: Error finding container bb0055b9b61dd03e030cde46ad02f37e072d395f72b57e4e12c52c0ab1605c60: Status 404 returned error can't find the container with id bb0055b9b61dd03e030cde46ad02f37e072d395f72b57e4e12c52c0ab1605c60 Jul 14 10:00:17.092976 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.092958 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:17.106243 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.106215 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qnx4d" Jul 14 10:00:17.111999 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.111975 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" Jul 14 10:00:17.113192 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.113174 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod888bf1ef_3b9d_4eb9_b4cc_795798943c72.slice/crio-23f8f99ab2090bbe4a934bbfbd0841588257d0ff594ea65f669ce6eca51e3f41 WatchSource:0}: Error finding container 23f8f99ab2090bbe4a934bbfbd0841588257d0ff594ea65f669ce6eca51e3f41: Status 404 returned error can't find the container with id 23f8f99ab2090bbe4a934bbfbd0841588257d0ff594ea65f669ce6eca51e3f41 Jul 14 10:00:17.119450 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.119423 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39181225_4694_44bb_9536_ea89b9880b1a.slice/crio-60826eb3d49536c441a5ba2afa6b29c9695f2851651bbb2aeaf32913f51422ef WatchSource:0}: Error finding container 60826eb3d49536c441a5ba2afa6b29c9695f2851651bbb2aeaf32913f51422ef: Status 404 returned error can't find the container with id 60826eb3d49536c441a5ba2afa6b29c9695f2851651bbb2aeaf32913f51422ef Jul 14 10:00:17.134371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.134349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6npvl" Jul 14 10:00:17.141048 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.141021 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37855e49_213f_467c_8c14_40ed6438305a.slice/crio-337ffe512466a04b9a5f9562ea533ebabdf58e5e69bb98521cde1ffd36c5820c WatchSource:0}: Error finding container 337ffe512466a04b9a5f9562ea533ebabdf58e5e69bb98521cde1ffd36c5820c: Status 404 returned error can't find the container with id 337ffe512466a04b9a5f9562ea533ebabdf58e5e69bb98521cde1ffd36c5820c Jul 14 10:00:17.152528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.152507 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" Jul 14 10:00:17.158821 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.158803 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r6dc4" Jul 14 10:00:17.159595 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.159576 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f3f6e5a_125e_47cb_805a_bd114123f2bc.slice/crio-0b3174237ea5c0e35ec62d30f9f5f18c1190a6037cd90ad56b462fbc20f74530 WatchSource:0}: Error finding container 0b3174237ea5c0e35ec62d30f9f5f18c1190a6037cd90ad56b462fbc20f74530: Status 404 returned error can't find the container with id 0b3174237ea5c0e35ec62d30f9f5f18c1190a6037cd90ad56b462fbc20f74530 Jul 14 10:00:17.179266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.179245 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-khgqj" Jul 14 10:00:17.183515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.183495 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:17.187133 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.187105 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee471c5_e56c_45c5_b717_c690e2d36229.slice/crio-861011bf9717c14e00156c6626c6bbe5b42498e0bbfaa1a2860773db0fbc87b5 WatchSource:0}: Error finding container 861011bf9717c14e00156c6626c6bbe5b42498e0bbfaa1a2860773db0fbc87b5: Status 404 returned error can't find the container with id 861011bf9717c14e00156c6626c6bbe5b42498e0bbfaa1a2860773db0fbc87b5 Jul 14 10:00:17.192998 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:17.192977 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7773ae2_7435_415e_8bbf_7fa42bdb462e.slice/crio-e81d62d7d54e3e19262f6c4b644315ab2eee65c96e102ad41e17dbc54d65813d WatchSource:0}: Error finding container e81d62d7d54e3e19262f6c4b644315ab2eee65c96e102ad41e17dbc54d65813d: Status 404 returned error can't find the container with id e81d62d7d54e3e19262f6c4b644315ab2eee65c96e102ad41e17dbc54d65813d Jul 14 10:00:17.447552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.447460 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:17.447552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.447507 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:17.447768 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.447608 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:17.447768 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.447650 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:17.447768 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.447663 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:18.447646816 +0000 UTC m=+3.081710635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:17.447768 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.447721 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:18.447703644 +0000 UTC m=+3.081767470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:17.548166 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.548119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:17.548347 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.548316 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:17.548347 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.548336 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:17.548451 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.548349 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:17.548451 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.548408 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:18.548388327 +0000 UTC m=+3.182452153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:17.634415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.634384 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:17.775139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.775013 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:55:16 +0000 UTC" deadline="2028-03-31 15:51:29.176259008 +0000 UTC" Jul 14 10:00:17.775139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.775076 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15029h51m11.401188106s" Jul 14 10:00:17.868376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.867592 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:17.868376 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.867718 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:17.868376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.868100 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:17.868376 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:17.868217 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:17.888761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.888572 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"a2392c717318c4fd2e490acb63a2f320f3fcbebc29ad2aa5faf7971e78630bb1"} Jul 14 10:00:17.901641 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.901601 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" event={"ID":"01c463973672e11a273c693fb7db70e9","Type":"ContainerStarted","Data":"698cc2ad9f64ff37b83d686551c15e70a22e13f3298add87a8cda576d9fc239c"} Jul 14 10:00:17.903587 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.903563 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r6dc4" event={"ID":"d27d3101-05df-4486-a7e4-1dcb0fc993bc","Type":"ContainerStarted","Data":"3798d8083dde114030a971581df64ff9cd73c7c8c4dc93c4f574921d8666472a"} Jul 14 10:00:17.905869 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.905843 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" event={"ID":"9f3f6e5a-125e-47cb-805a-bd114123f2bc","Type":"ContainerStarted","Data":"0b3174237ea5c0e35ec62d30f9f5f18c1190a6037cd90ad56b462fbc20f74530"} Jul 14 10:00:17.911373 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.911344 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vm4nd" event={"ID":"71976dd2-2c7d-406a-8e1a-ffc873e42174","Type":"ContainerStarted","Data":"bb0055b9b61dd03e030cde46ad02f37e072d395f72b57e4e12c52c0ab1605c60"} Jul 14 10:00:17.915616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.915583 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tjjsb" event={"ID":"f7773ae2-7435-415e-8bbf-7fa42bdb462e","Type":"ContainerStarted","Data":"e81d62d7d54e3e19262f6c4b644315ab2eee65c96e102ad41e17dbc54d65813d"} Jul 14 10:00:17.921770 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.921734 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-khgqj" event={"ID":"4ee471c5-e56c-45c5-b717-c690e2d36229","Type":"ContainerStarted","Data":"861011bf9717c14e00156c6626c6bbe5b42498e0bbfaa1a2860773db0fbc87b5"} Jul 14 10:00:17.927532 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.927460 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6npvl" event={"ID":"37855e49-213f-467c-8c14-40ed6438305a","Type":"ContainerStarted","Data":"337ffe512466a04b9a5f9562ea533ebabdf58e5e69bb98521cde1ffd36c5820c"} Jul 14 10:00:17.940225 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.940189 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerStarted","Data":"60826eb3d49536c441a5ba2afa6b29c9695f2851651bbb2aeaf32913f51422ef"} Jul 14 10:00:17.948983 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:17.948947 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnx4d" event={"ID":"888bf1ef-3b9d-4eb9-b4cc-795798943c72","Type":"ContainerStarted","Data":"23f8f99ab2090bbe4a934bbfbd0841588257d0ff594ea65f669ce6eca51e3f41"} Jul 14 10:00:18.456003 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.455957 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:18.456216 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.456026 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:18.456216 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.456195 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:18.456329 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.456267 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:20.456247643 +0000 UTC m=+5.090311465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:18.456715 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.456688 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:18.456804 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.456740 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:20.456725449 +0000 UTC m=+5.090789268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:18.557743 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.557713 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:18.557743 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.557749 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:18.557978 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.557762 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:18.557978 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.557819 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:20.557801099 +0000 UTC m=+5.191864936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:18.558286 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.558220 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:18.775681 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.775569 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 09:55:16 +0000 UTC" deadline="2028-01-05 23:48:46.550509019 +0000 UTC" Jul 14 10:00:18.775681 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.775653 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12973h48m27.774860365s" Jul 14 10:00:18.865059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.864598 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:18.865059 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:18.864726 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:18.980266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:18.980227 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" event={"ID":"5af106930072935ac7f96a0491100845","Type":"ContainerStarted","Data":"36cf8dbc3a28c730ea28ad2ad20a28ff52c2bc0e47db79144e0194a1a4bc8c42"} Jul 14 10:00:19.474093 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:19.472708 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 10:00:19.867459 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:19.867403 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:19.867884 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:19.867524 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:19.867947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:19.867917 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:19.868054 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:19.868033 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:20.474191 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:20.474252 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.474398 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.474454 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:24.474436566 +0000 UTC m=+9.108500390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.474516 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:20.474861 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.474546 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:24.474533583 +0000 UTC m=+9.108597403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:20.574855 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:20.574818 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:20.575032 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.574969 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:20.575032 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.574990 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:20.575032 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.575002 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:20.575230 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.575061 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:24.575043015 +0000 UTC m=+9.209106840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:20.865279 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:20.865201 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:20.865411 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:20.865334 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:21.865503 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:21.865469 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:21.865884 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:21.865621 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:21.866228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:21.866061 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:21.866228 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:21.866194 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:22.864570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:22.864524 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:22.864745 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:22.864663 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:23.865043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:23.864965 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:23.865496 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:23.865103 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:23.865496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:23.865308 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:23.865496 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:23.865412 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:24.505766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:24.505728 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:24.505960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:24.505821 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:24.506021 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.506009 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:24.506077 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.506070 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:32.506049986 +0000 UTC m=+17.140113810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:24.506344 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.506317 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:24.506441 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.506389 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:32.506369233 +0000 UTC m=+17.140433063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:24.607273 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:24.607170 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:24.607457 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.607369 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:24.607457 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.607396 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:24.607457 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.607411 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:24.607614 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.607482 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:32.607462026 +0000 UTC m=+17.241525853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:24.865388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:24.865286 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:24.865851 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:24.865418 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:25.866512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:25.866025 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:25.866512 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:25.866131 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:25.866512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:25.866179 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:25.866512 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:25.866311 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:26.865401 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:26.865357 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:26.865586 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:26.865532 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:27.864782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:27.864746 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:27.865125 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:27.864891 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:27.865125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:27.864935 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:27.865125 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:27.865017 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:28.865248 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:28.865217 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:28.865615 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:28.865321 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:29.865020 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:29.864986 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:29.865217 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:29.865125 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:29.865217 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:29.865206 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:29.865640 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:29.865292 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:30.865047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:30.865011 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:30.865244 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:30.865136 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:31.865292 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:31.865259 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:31.865753 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:31.865266 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:31.865753 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:31.865399 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:31.865753 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:31.865460 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:32.568260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:32.568231 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:32.568516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:32.568300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:32.568516 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.568401 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:32.568516 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.568490 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:48.568468703 +0000 UTC m=+33.202532539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:32.568516 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.568406 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:32.568734 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.568570 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:48.568552979 +0000 UTC m=+33.202616811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:32.668545 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:32.668517 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:32.668694 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.668660 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:32.668694 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.668678 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:32.668694 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.668687 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:32.668807 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.668734 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:00:48.668721255 +0000 UTC m=+33.302785073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:32.864824 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:32.864751 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:32.864968 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:32.864862 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:33.864529 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:33.864498 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:33.865027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:33.864498 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:33.865027 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:33.864636 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:33.865027 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:33.864675 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:34.865475 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:34.865441 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:34.865931 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:34.865575 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:35.865690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:35.865465 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:35.866132 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:35.865545 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:35.866132 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:35.865778 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:35.866132 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:35.865855 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:36.864800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:36.864765 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:36.864954 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:36.864871 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:37.865611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:37.865584 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:37.866090 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:37.865623 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:37.866090 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:37.865712 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:37.866090 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:37.865806 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:38.031785 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.031745 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6npvl" event={"ID":"37855e49-213f-467c-8c14-40ed6438305a","Type":"ContainerStarted","Data":"e5ba7f3b641f0faa9fd4fdd868e8e8d012dbd82e9993df14c5eeb305c8196a9c"} Jul 14 10:00:38.034765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.034745 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:00:38.035139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035117 2569 generic.go:356] "Generic (PLEG): container finished" podID="784d21c1-2b89-4930-acae-af220fb6fe25" containerID="688c20c5b9aff2dd4423d5e9eed1b7ce572b06b7c03ae1c7dade2e0f5eee9fe0" exitCode=1 Jul 14 10:00:38.035258 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035184 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"25cbff771a603578820da564c6537dfcb6d0648f428715b93a14d6451640974f"} Jul 14 10:00:38.035325 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035266 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"82ca20180f7642310a3463feef71ae4c002ea27ff7f83f5a82dea6a9ac2c38b3"} Jul 14 10:00:38.035325 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035318 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"2d063646a88a0022b56e3c62c91f8ab80c501b67630d1bbd231a8519bc9d3a09"} Jul 14 10:00:38.035425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035364 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"d5a024e96b9ff633a43483f03af51c98c121cbb31f3d3150465eeda1ca19832e"} Jul 14 10:00:38.035425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035409 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerDied","Data":"688c20c5b9aff2dd4423d5e9eed1b7ce572b06b7c03ae1c7dade2e0f5eee9fe0"} Jul 14 10:00:38.035524 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.035459 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"8fe0b4dece5a25ddaddfb678dab0b8b85d9c87351c829140205ce39ca931bc5b"} Jul 14 10:00:38.037011 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.036968 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r6dc4" event={"ID":"d27d3101-05df-4486-a7e4-1dcb0fc993bc","Type":"ContainerStarted","Data":"ebb0894a2b46042be25d1ed0389aa70b2c86511a8b37472a3449367914640c3e"} Jul 14 10:00:38.038522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.038498 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" event={"ID":"5af106930072935ac7f96a0491100845","Type":"ContainerStarted","Data":"9b9d7bce4044f8acbfcce76a3ffcc41bae7b8c68c72114dff08854a86938f04e"} Jul 14 10:00:38.048586 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.048544 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6npvl" podStartSLOduration=2.881848413 podStartE2EDuration="23.048525468s" totalImagesPullingTime="20.166677055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:38.048424268 +0000 UTC m=+22.682488108" watchObservedRunningTime="2026-07-14 10:00:38.048525468 +0000 UTC m=+22.682589313" Jul 14 10:00:38.061743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.061703 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-78.ec2.internal" podStartSLOduration=22.061690772 podStartE2EDuration="22.061690772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:38.061478522 +0000 UTC m=+22.695542362" watchObservedRunningTime="2026-07-14 10:00:38.061690772 +0000 UTC m=+22.695754612" Jul 14 10:00:38.078901 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.078854 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r6dc4" podStartSLOduration=2.936952597 podStartE2EDuration="23.078840506s" totalImagesPullingTime="20.141887909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:38.07857491 +0000 UTC m=+22.712638751" watchObservedRunningTime="2026-07-14 10:00:38.078840506 +0000 UTC m=+22.712904347" Jul 14 10:00:38.865120 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:38.864804 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:38.865278 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:38.865231 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:39.041834 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.041800 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tjjsb" event={"ID":"f7773ae2-7435-415e-8bbf-7fa42bdb462e","Type":"ContainerStarted","Data":"004e68265387b62c47ac7d9bd0ef0f884429ce93f32dd1052db2daa78026e910"} Jul 14 10:00:39.043261 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.043234 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-khgqj" event={"ID":"4ee471c5-e56c-45c5-b717-c690e2d36229","Type":"ContainerStarted","Data":"b2f4599b235cf4580b543371160a4f1dab0a4710221a88d981d1907d8832ec20"} Jul 14 10:00:39.047329 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.047304 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="22811ca84e08e81f88c71a22a25b23da67193da418f119e9f8c29fb02beac567" exitCode=0 Jul 14 10:00:39.047433 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.047382 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"22811ca84e08e81f88c71a22a25b23da67193da418f119e9f8c29fb02beac567"} Jul 14 10:00:39.048774 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.048751 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qnx4d" event={"ID":"888bf1ef-3b9d-4eb9-b4cc-795798943c72","Type":"ContainerStarted","Data":"7057c790dcb2093779a03ec49ba307df0bbd231ad88de391f5eda93dfd0abf4f"} Jul 14 10:00:39.050228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.050209 2569 generic.go:356] "Generic (PLEG): container finished" podID="01c463973672e11a273c693fb7db70e9" containerID="ecf66cc0fca6c86276b3afc0940c0b180f0912be72f172cb11a57581a57a632f" exitCode=0 Jul 14 10:00:39.050335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.050284 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" event={"ID":"01c463973672e11a273c693fb7db70e9","Type":"ContainerDied","Data":"ecf66cc0fca6c86276b3afc0940c0b180f0912be72f172cb11a57581a57a632f"} Jul 14 10:00:39.051590 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.051573 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" event={"ID":"9f3f6e5a-125e-47cb-805a-bd114123f2bc","Type":"ContainerStarted","Data":"9d5516006d4c3028f313a17bb060d4c2301695a142dcf4654396d7173dee0127"} Jul 14 10:00:39.052915 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.052884 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vm4nd" event={"ID":"71976dd2-2c7d-406a-8e1a-ffc873e42174","Type":"ContainerStarted","Data":"848852cd62f40a36f119d7e4644db80e0a19af211780d2012be470c7f7569ca9"} Jul 14 10:00:39.070899 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.070861 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tjjsb" podStartSLOduration=11.211955159 podStartE2EDuration="23.070851015s" totalImagesPullingTime="11.858895856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:39.070596561 +0000 UTC m=+23.704660402" watchObservedRunningTime="2026-07-14 10:00:39.070851015 +0000 UTC m=+23.704914856" Jul 14 10:00:39.085261 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.085225 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-khgqj" podStartSLOduration=4.101132897 podStartE2EDuration="24.085214044s" totalImagesPullingTime="19.984081147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:39.085056056 +0000 UTC m=+23.719119898" watchObservedRunningTime="2026-07-14 10:00:39.085214044 +0000 UTC m=+23.719277887" Jul 14 10:00:39.115689 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.115653 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vm4nd" podStartSLOduration=4.031734091 podStartE2EDuration="24.115640681s" totalImagesPullingTime="20.08390659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:39.115113122 +0000 UTC m=+23.749176967" watchObservedRunningTime="2026-07-14 10:00:39.115640681 +0000 UTC m=+23.749704522" Jul 14 10:00:39.129463 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.129424 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qnx4d" podStartSLOduration=3.935435569 podStartE2EDuration="24.129411995s" totalImagesPullingTime="20.193976426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:39.129268432 +0000 UTC m=+23.763332272" watchObservedRunningTime="2026-07-14 10:00:39.129411995 +0000 UTC m=+23.763475877" Jul 14 10:00:39.865048 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.865021 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:39.865048 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:39.865047 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:39.865329 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:39.865181 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:39.865329 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:39.865183 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:40.058472 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.058004 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" event={"ID":"01c463973672e11a273c693fb7db70e9","Type":"ContainerStarted","Data":"46195e4fdec9dbee1b3e2a3c8ea061e916c4ff045bf73470829536ffdc7111d7"} Jul 14 10:00:40.074964 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.074913 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-78.ec2.internal" podStartSLOduration=24.074894954 podStartE2EDuration="24.074894954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:40.07408786 +0000 UTC m=+24.708151702" watchObservedRunningTime="2026-07-14 10:00:40.074894954 +0000 UTC m=+24.708958800" Jul 14 10:00:40.155618 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.155595 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 10:00:40.801144 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.801007 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T10:00:40.155613334Z","UUID":"ac8c85fb-1f8d-4722-9c50-af60542479a9","Handler":null,"Name":"","Endpoint":""} Jul 14 10:00:40.803000 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.802976 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 10:00:40.803109 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.803027 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 10:00:40.865046 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:40.865016 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:40.865228 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:40.865140 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:41.063098 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:41.063024 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:00:41.063515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:41.063453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"768550ecc7aaff55ddc14e146bf81f89645e9d9392cc2c6ac9e54b69ee8f6c3f"} Jul 14 10:00:41.065215 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:41.065185 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" event={"ID":"9f3f6e5a-125e-47cb-805a-bd114123f2bc","Type":"ContainerStarted","Data":"e6f98cd43574a5a40ef99dff17fe5da6efc5f4b93b3d934baa491574a9754c33"} Jul 14 10:00:41.865471 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:41.865437 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:41.865656 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:41.865583 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:41.865656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:41.865643 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:41.865784 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:41.865762 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:42.070134 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:42.070099 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" event={"ID":"9f3f6e5a-125e-47cb-805a-bd114123f2bc","Type":"ContainerStarted","Data":"65c37f001e9c493eec14ca16969b5ac3a9b8f5017be7cbfdb60aa4b178fc901f"} Jul 14 10:00:42.088662 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:42.088607 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2q547" podStartSLOduration=4.238824306 podStartE2EDuration="27.088593063s" totalImagesPullingTime="22.849768757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:42.088276872 +0000 UTC m=+26.722340712" watchObservedRunningTime="2026-07-14 10:00:42.088593063 +0000 UTC m=+26.722656903" Jul 14 10:00:42.643203 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:42.643170 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:42.643839 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:42.643818 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:42.865256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:42.865186 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:42.865414 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:42.865287 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:43.073321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:43.073289 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:43.073765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:43.073693 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tjjsb" Jul 14 10:00:43.865238 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:43.865053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:43.865327 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:43.865053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:43.865376 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:43.865340 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:43.865423 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:43.865396 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:44.077786 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.077762 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:00:44.078553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.078120 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"7928e796cf81a024b6dd39e978fde4a7b5ef2d7c2c901ff9d0991da3dbcf6994"} Jul 14 10:00:44.078553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.078350 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:44.078643 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.078625 2569 scope.go:117] "RemoveContainer" containerID="688c20c5b9aff2dd4423d5e9eed1b7ce572b06b7c03ae1c7dade2e0f5eee9fe0" Jul 14 10:00:44.080409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.080383 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="4118147f7fe6343afa5e60e0bb3222833b105b0d7bad3eebd68f0da3e8dcaa78" exitCode=0 Jul 14 10:00:44.080513 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.080470 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"4118147f7fe6343afa5e60e0bb3222833b105b0d7bad3eebd68f0da3e8dcaa78"} Jul 14 10:00:44.093915 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.093897 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:44.866329 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:44.866059 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:44.866501 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:44.866429 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:45.085922 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.085898 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:00:45.086373 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.086286 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" event={"ID":"784d21c1-2b89-4930-acae-af220fb6fe25","Type":"ContainerStarted","Data":"82b3a5ce47cfa11deacee7c5f67a29e4800a8f697270808f45d6dd2949a3321d"} Jul 14 10:00:45.086453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.086378 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 14 10:00:45.086688 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.086669 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:45.102287 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.102264 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:45.120451 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.120375 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" podStartSLOduration=9.813356853 podStartE2EDuration="30.120363415s" totalImagesPullingTime="20.307006562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:45.120126682 +0000 UTC m=+29.754190524" watchObservedRunningTime="2026-07-14 10:00:45.120363415 +0000 UTC m=+29.754427256" Jul 14 10:00:45.269957 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.269932 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:00:45.624430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.624400 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7nmln"] Jul 14 10:00:45.624621 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.624502 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:45.624621 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:45.624576 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:45.627293 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.627271 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dgbms"] Jul 14 10:00:45.627407 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.627379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:45.627509 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:45.627489 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:45.627788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.627768 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f9qfj"] Jul 14 10:00:45.627866 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:45.627854 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:45.627967 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:45.627926 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:46.864849 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:46.864772 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:46.865211 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:46.864861 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:46.865211 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:46.864878 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:46.865211 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:46.864934 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:46.865211 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:46.864966 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:46.865211 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:46.865055 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:47.092732 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:47.092703 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="444b4aa91e2cec8a143c7deabc7e09249487a53dc264062925e16cacd2de6be3" exitCode=0 Jul 14 10:00:47.092880 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:47.092784 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"444b4aa91e2cec8a143c7deabc7e09249487a53dc264062925e16cacd2de6be3"} Jul 14 10:00:48.097488 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.097270 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="1f3ec6bd03fd63648b296d7671c5199bf478ece06f435417da11baa5be6496ff" exitCode=0 Jul 14 10:00:48.097488 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.097348 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"1f3ec6bd03fd63648b296d7671c5199bf478ece06f435417da11baa5be6496ff"} Jul 14 10:00:48.599565 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.599536 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:48.599711 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.599577 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:48.599711 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.599661 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:48.599711 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.599683 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:48.599711 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.599710 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret podName:b68475b9-1f2f-4ad9-875c-ea54c2400cc0 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:20.599695805 +0000 UTC m=+65.233759623 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret") pod "global-pull-secret-syncer-7nmln" (UID: "b68475b9-1f2f-4ad9-875c-ea54c2400cc0") : object "kube-system"/"original-pull-secret" not registered Jul 14 10:00:48.599876 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.599736 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:20.599723258 +0000 UTC m=+65.233787081 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 10:00:48.700030 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.700004 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:48.700192 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.700115 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 10:00:48.700192 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.700127 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 10:00:48.700192 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.700136 2569 projected.go:196] Error preparing data for projected volume kube-api-access-spbb4 for pod openshift-network-diagnostics/network-check-target-f9qfj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:48.700311 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.700195 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4 podName:6ba17703-e928-47bf-9518-f92ac981131e nodeName:}" failed. No retries permitted until 2026-07-14 10:01:20.700182203 +0000 UTC m=+65.334246021 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-spbb4" (UniqueName: "kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4") pod "network-check-target-f9qfj" (UID: "6ba17703-e928-47bf-9518-f92ac981131e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 10:00:48.865039 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.865005 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:48.865227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.865049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:48.865227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:48.865049 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:48.865227 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.865129 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dgbms" podUID="fcdbec16-4a3f-4206-a79f-cf02af0a3b17" Jul 14 10:00:48.865363 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.865243 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7nmln" podUID="b68475b9-1f2f-4ad9-875c-ea54c2400cc0" Jul 14 10:00:48.865363 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:48.865308 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9qfj" podUID="6ba17703-e928-47bf-9518-f92ac981131e" Jul 14 10:00:50.740606 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.740567 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-78.ec2.internal" event="NodeReady" Jul 14 10:00:50.741111 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.740723 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 10:00:50.775789 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.775763 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:00:50.795355 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.795331 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g72sn"] Jul 14 10:00:50.795519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.795474 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.798377 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.798356 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 10:00:50.798576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.798554 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 10:00:50.798760 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.798632 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 10:00:50.799509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.799487 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zh9t2\"" Jul 14 10:00:50.804570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.804548 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 10:00:50.813459 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.813440 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-chm6b"] Jul 14 10:00:50.813586 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.813570 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:50.816090 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.816073 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d25fg\"" Jul 14 10:00:50.816201 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.816072 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 10:00:50.816267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.816080 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 10:00:50.835162 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.835126 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:00:50.835272 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.835176 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g72sn"] Jul 14 10:00:50.835272 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.835210 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-chm6b"] Jul 14 10:00:50.835377 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.835306 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:50.837764 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.837727 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 10:00:50.838062 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.838037 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 10:00:50.838062 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.838055 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 10:00:50.838249 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.838141 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-26p75\"" Jul 14 10:00:50.865244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.865220 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:00:50.865244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.865234 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:00:50.865244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.865236 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:00:50.869589 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.869565 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 10:00:50.869988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.869969 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pz2v6\"" Jul 14 10:00:50.870184 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.870144 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 10:00:50.870752 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.870731 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 10:00:50.870842 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.870821 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 10:00:50.870909 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.870845 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vm2xk\"" Jul 14 10:00:50.916466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916431 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzhpj\" (UniqueName: \"kubernetes.io/projected/c8013d14-4407-485c-8817-c2bb32a9a7a0-kube-api-access-fzhpj\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:50.916628 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:50.916628 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916499 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9ad799-4001-448a-8947-36def65e3f94-tmp-dir\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:50.916628 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916523 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916787 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916629 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916787 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916691 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f829h\" (UniqueName: \"kubernetes.io/projected/2b9ad799-4001-448a-8947-36def65e3f94-kube-api-access-f829h\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:50.916787 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916718 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwxfp\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916787 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9ad799-4001-448a-8947-36def65e3f94-config-volume\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:50.916946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916789 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916851 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:50.916946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916935 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8013d14-4407-485c-8817-c2bb32a9a7a0-cert\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:50.917130 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:50.916961 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.017782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.017705 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.017782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.017748 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9ad799-4001-448a-8947-36def65e3f94-tmp-dir\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.017782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.017772 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018028 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.017877 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:00:51.018028 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.017950 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:51.517925809 +0000 UTC m=+36.151989635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:00:51.018137 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018053 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018137 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018112 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f829h\" (UniqueName: \"kubernetes.io/projected/2b9ad799-4001-448a-8947-36def65e3f94-kube-api-access-f829h\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018137 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwxfp\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018179 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9ad799-4001-448a-8947-36def65e3f94-tmp-dir\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018188 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9ad799-4001-448a-8947-36def65e3f94-config-volume\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018212 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018242 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018278 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018264 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018631 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018631 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018609 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8013d14-4407-485c-8817-c2bb32a9a7a0-cert\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:51.018735 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018647 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.018735 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018697 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzhpj\" (UniqueName: \"kubernetes.io/projected/c8013d14-4407-485c-8817-c2bb32a9a7a0-kube-api-access-fzhpj\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:51.018735 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018695 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b9ad799-4001-448a-8947-36def65e3f94-config-volume\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.018878 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.018812 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:00:51.018878 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.018826 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:00:51.018878 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.018875 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:00:51.518861918 +0000 UTC m=+36.152925751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:00:51.019064 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.018948 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.019204 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.019183 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.022745 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.022701 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.022847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.022753 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8013d14-4407-485c-8817-c2bb32a9a7a0-cert\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:51.022847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.022753 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.027276 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.027252 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzhpj\" (UniqueName: \"kubernetes.io/projected/c8013d14-4407-485c-8817-c2bb32a9a7a0-kube-api-access-fzhpj\") pod \"ingress-canary-chm6b\" (UID: \"c8013d14-4407-485c-8817-c2bb32a9a7a0\") " pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:51.027607 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.027565 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f829h\" (UniqueName: \"kubernetes.io/projected/2b9ad799-4001-448a-8947-36def65e3f94-kube-api-access-f829h\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.028031 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.028008 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.028105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.028059 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwxfp\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.030784 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.030758 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.145001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.144975 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-chm6b" Jul 14 10:00:51.523766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.523727 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:51.523961 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:51.523797 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:51.523961 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.523898 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:00:51.524059 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.523987 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:52.523966268 +0000 UTC m=+37.158030096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:00:51.524059 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.523900 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:00:51.524059 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.524008 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:00:51.524205 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:51.524079 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:00:52.524037878 +0000 UTC m=+37.158101701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:00:52.531370 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:52.531328 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:52.531445 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:52.531511 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:52.531562 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:52.531577 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:52.531583 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:54.531564092 +0000 UTC m=+39.165627920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:00:52.531965 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:52.531626 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:00:54.531610723 +0000 UTC m=+39.165674580 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:00:52.845493 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:52.845463 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-chm6b"] Jul 14 10:00:52.910525 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:00:52.910491 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8013d14_4407_485c_8817_c2bb32a9a7a0.slice/crio-3a5a8a9c4ea08c0120ce453e4801254e8d7045e88236284e7eac1d608ab3d644 WatchSource:0}: Error finding container 3a5a8a9c4ea08c0120ce453e4801254e8d7045e88236284e7eac1d608ab3d644: Status 404 returned error can't find the container with id 3a5a8a9c4ea08c0120ce453e4801254e8d7045e88236284e7eac1d608ab3d644 Jul 14 10:00:53.111690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:53.111480 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerStarted","Data":"35add2124e578a32fbf46cd6cc14a42a61f534e4232ba97ab1135270d348ed05"} Jul 14 10:00:53.112611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:53.112587 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-chm6b" event={"ID":"c8013d14-4407-485c-8817-c2bb32a9a7a0","Type":"ContainerStarted","Data":"3a5a8a9c4ea08c0120ce453e4801254e8d7045e88236284e7eac1d608ab3d644"} Jul 14 10:00:54.118446 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:54.118417 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="35add2124e578a32fbf46cd6cc14a42a61f534e4232ba97ab1135270d348ed05" exitCode=0 Jul 14 10:00:54.118993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:54.118454 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"35add2124e578a32fbf46cd6cc14a42a61f534e4232ba97ab1135270d348ed05"} Jul 14 10:00:54.545922 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:54.545880 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:54.546083 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:54.545938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:54.546083 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:54.546040 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:00:54.546227 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:54.546112 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:00:58.546097422 +0000 UTC m=+43.180161241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:00:54.546227 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:54.546043 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:00:54.546227 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:54.546165 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:00:54.546227 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:54.546211 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:00:58.546198711 +0000 UTC m=+43.180262535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:00:55.125335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:55.125295 2569 generic.go:356] "Generic (PLEG): container finished" podID="39181225-4694-44bb-9536-ea89b9880b1a" containerID="6effc41face299535450dda0d87001dbd49287efc3cef99d16145fe8dbd1876c" exitCode=0 Jul 14 10:00:55.125760 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:55.125345 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerDied","Data":"6effc41face299535450dda0d87001dbd49287efc3cef99d16145fe8dbd1876c"} Jul 14 10:00:56.128868 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:56.128831 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-chm6b" event={"ID":"c8013d14-4407-485c-8817-c2bb32a9a7a0","Type":"ContainerStarted","Data":"72ba72c600cc806fe3da557e6508805d00be4f6f41b6f37bc908fb03fceb5d87"} Jul 14 10:00:56.131928 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:56.131893 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" event={"ID":"39181225-4694-44bb-9536-ea89b9880b1a","Type":"ContainerStarted","Data":"062b2363eebe6a0959c39e4d80f8ea339ff3709ca896acc9fc691dbde90ba503"} Jul 14 10:00:56.144201 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:56.144147 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-chm6b" podStartSLOduration=3.488121864 podStartE2EDuration="6.14413328s" totalImagesPullingTime="2.656011416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:56.143518376 +0000 UTC m=+40.777582228" watchObservedRunningTime="2026-07-14 10:00:56.14413328 +0000 UTC m=+40.778197141" Jul 14 10:00:56.164378 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:56.164339 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hqpdc" podStartSLOduration=8.579253629 podStartE2EDuration="41.164326732s" totalImagesPullingTime="32.585073103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:00:56.163953313 +0000 UTC m=+40.798017165" watchObservedRunningTime="2026-07-14 10:00:56.164326732 +0000 UTC m=+40.798390573" Jul 14 10:00:58.577126 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:58.577085 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:00:58.577140 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:58.577250 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:58.577252 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:58.577271 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:58.577301 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:06.57728719 +0000 UTC m=+51.211351008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:00:58.577507 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:00:58.577313 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:01:06.57730792 +0000 UTC m=+51.211371739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:01:05.314558 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.314524 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vpnjq"] Jul 14 10:01:05.327998 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.327970 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vpnjq"] Jul 14 10:01:05.328190 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.328036 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:05.334086 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.334057 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 10:01:05.334086 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.334074 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 10:01:05.335246 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.335227 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tn8fb\"" Jul 14 10:01:05.428764 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.428718 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:05.428764 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.428768 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:05.529898 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.529861 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:05.530044 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.529914 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:05.530044 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:05.530011 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:05.530123 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:05.530072 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:06.030058376 +0000 UTC m=+50.664122195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:05.530524 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:05.530506 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:06.033691 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.033636 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:06.033907 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.033816 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:06.033907 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.033905 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:07.033882115 +0000 UTC m=+51.667945933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:06.478749 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.478716 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bwk7v"] Jul 14 10:01:06.492315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.492282 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bwk7v"] Jul 14 10:01:06.492498 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.492425 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.494903 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.494877 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 10:01:06.495036 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.494877 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 10:01:06.495036 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.494879 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 10:01:06.495972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.495954 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 10:01:06.496061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.495970 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-szrgf\"" Jul 14 10:01:06.536887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.536849 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b0e354f-e5d2-4f1b-9324-2912327875b1-data-volume\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.536887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.536903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.537144 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.536952 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b0e354f-e5d2-4f1b-9324-2912327875b1-crio-socket\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.537144 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.537046 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.537144 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.537073 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwtzt\" (UniqueName: \"kubernetes.io/projected/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-api-access-hwtzt\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638232 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwtzt\" (UniqueName: \"kubernetes.io/projected/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-api-access-hwtzt\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638300 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b0e354f-e5d2-4f1b-9324-2912327875b1-data-volume\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638322 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638349 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:06.638376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638363 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b0e354f-e5d2-4f1b-9324-2912327875b1-crio-socket\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638556 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638472 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:06.638556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638526 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:06.638556 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638551 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:07.138529125 +0000 UTC m=+51.772592956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638559 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b0e354f-e5d2-4f1b-9324-2912327875b1-crio-socket\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638475 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638596 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638599 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9b4d7f958-n5tfw: secret "image-registry-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638640 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls podName:2b9ad799-4001-448a-8947-36def65e3f94 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:22.638628848 +0000 UTC m=+67.272692667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls") pod "dns-default-g72sn" (UID: "2b9ad799-4001-448a-8947-36def65e3f94") : secret "dns-default-metrics-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638650 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638656 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls podName:bcb4c5da-0352-404b-9005-b9a2ea7fdf6d nodeName:}" failed. No retries permitted until 2026-07-14 10:01:22.638647537 +0000 UTC m=+67.272711357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls") pod "image-registry-9b4d7f958-n5tfw" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d") : secret "image-registry-tls" not found Jul 14 10:01:06.638701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.638698 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b0e354f-e5d2-4f1b-9324-2912327875b1-data-volume\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:06.639000 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:06.638721 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:07.138703598 +0000 UTC m=+51.772767419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : configmap "kube-rbac-proxy" not found Jul 14 10:01:06.651390 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:06.651363 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwtzt\" (UniqueName: \"kubernetes.io/projected/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-api-access-hwtzt\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:07.042470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:07.042426 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:07.042657 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.042581 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:07.042657 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.042644 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:09.042628488 +0000 UTC m=+53.676692308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:07.143111 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:07.143075 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:07.143224 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:07.143147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:07.143267 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.143225 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 10:01:07.143267 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.143262 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:07.143334 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.143298 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:08.143282351 +0000 UTC m=+52.777346169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : configmap "kube-rbac-proxy" not found Jul 14 10:01:07.143334 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:07.143314 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:08.143308658 +0000 UTC m=+52.777372476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:08.151748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:08.151710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:08.152188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:08.151796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:08.152188 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:08.151878 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 10:01:08.152188 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:08.151880 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:08.152188 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:08.151940 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:10.151924441 +0000 UTC m=+54.785988260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : configmap "kube-rbac-proxy" not found Jul 14 10:01:08.152188 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:08.151953 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:10.151946715 +0000 UTC m=+54.786010534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:09.059534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:09.059492 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:09.059775 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:09.059647 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:09.059775 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:09.059722 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:13.059706287 +0000 UTC m=+57.693770109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:09.517787 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:09.517758 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vm4nd_71976dd2-2c7d-406a-8e1a-ffc873e42174/dns-node-resolver/0.log" Jul 14 10:01:10.167835 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:10.167803 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:10.168012 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:10.167949 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:10.168012 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:10.167966 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:10.168012 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:10.168000 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:14.167984584 +0000 UTC m=+58.802048404 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:10.168115 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:10.168022 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 10:01:10.168115 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:10.168078 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:14.168065336 +0000 UTC m=+58.802129156 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : configmap "kube-rbac-proxy" not found Jul 14 10:01:10.517614 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:10.517586 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-khgqj_4ee471c5-e56c-45c5-b717-c690e2d36229/node-ca/0.log" Jul 14 10:01:11.119312 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:11.119267 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-chm6b_c8013d14-4407-485c-8817-c2bb32a9a7a0/serve-healthcheck-canary/0.log" Jul 14 10:01:13.091081 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:13.091036 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:13.091470 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:13.091197 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:13.091470 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:13.091272 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:21.091256952 +0000 UTC m=+65.725320774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:14.200425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:14.200393 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:14.200850 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:14.200499 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:14.200850 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:14.200549 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:14.200850 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:14.200614 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:22.200600333 +0000 UTC m=+66.834664155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:14.200850 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:14.200642 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 10:01:14.200850 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:14.200716 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:22.200696325 +0000 UTC m=+66.834760161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : configmap "kube-rbac-proxy" not found Jul 14 10:01:17.103636 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:17.103601 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5zh8" Jul 14 10:01:20.649915 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.649862 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:01:20.650339 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.650031 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:01:20.652710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.652692 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 10:01:20.652762 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.652698 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 10:01:20.660907 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:20.660882 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 10:01:20.660972 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:20.660962 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs podName:fcdbec16-4a3f-4206-a79f-cf02af0a3b17 nodeName:}" failed. No retries permitted until 2026-07-14 10:02:24.660943188 +0000 UTC m=+129.295007007 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs") pod "network-metrics-daemon-dgbms" (UID: "fcdbec16-4a3f-4206-a79f-cf02af0a3b17") : secret "metrics-daemon-secret" not found Jul 14 10:01:20.663096 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.663071 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b68475b9-1f2f-4ad9-875c-ea54c2400cc0-original-pull-secret\") pod \"global-pull-secret-syncer-7nmln\" (UID: \"b68475b9-1f2f-4ad9-875c-ea54c2400cc0\") " pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:01:20.750741 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.750690 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:01:20.753345 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.753325 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 10:01:20.763680 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.763659 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 10:01:20.774265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.774229 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spbb4\" (UniqueName: \"kubernetes.io/projected/6ba17703-e928-47bf-9518-f92ac981131e-kube-api-access-spbb4\") pod \"network-check-target-f9qfj\" (UID: \"6ba17703-e928-47bf-9518-f92ac981131e\") " pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:01:20.879546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.879518 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pz2v6\"" Jul 14 10:01:20.887639 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.887617 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:01:20.890926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:20.890905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7nmln" Jul 14 10:01:21.024396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:21.024366 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f9qfj"] Jul 14 10:01:21.027855 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:21.027829 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba17703_e928_47bf_9518_f92ac981131e.slice/crio-96517885617fe00f475209b21b24f8251aa4fffaa54f0d888baea757a7cb9cd1 WatchSource:0}: Error finding container 96517885617fe00f475209b21b24f8251aa4fffaa54f0d888baea757a7cb9cd1: Status 404 returned error can't find the container with id 96517885617fe00f475209b21b24f8251aa4fffaa54f0d888baea757a7cb9cd1 Jul 14 10:01:21.034602 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:21.034580 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7nmln"] Jul 14 10:01:21.037555 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:21.037535 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb68475b9_1f2f_4ad9_875c_ea54c2400cc0.slice/crio-7417bd1a1c911c3b288908a99481df8c2830233ff4261352749e68908ceb2bb5 WatchSource:0}: Error finding container 7417bd1a1c911c3b288908a99481df8c2830233ff4261352749e68908ceb2bb5: Status 404 returned error can't find the container with id 7417bd1a1c911c3b288908a99481df8c2830233ff4261352749e68908ceb2bb5 Jul 14 10:01:21.153820 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:21.153707 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:21.153982 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:21.153855 2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 10:01:21.153982 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:21.153923 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert podName:d8da39dc-4ed7-4fd9-8221-6bc169ecb783 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:37.153908151 +0000 UTC m=+81.787971970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vpnjq" (UID: "d8da39dc-4ed7-4fd9-8221-6bc169ecb783") : secret "networking-console-plugin-cert" not found Jul 14 10:01:21.192677 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:21.192639 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7nmln" event={"ID":"b68475b9-1f2f-4ad9-875c-ea54c2400cc0","Type":"ContainerStarted","Data":"7417bd1a1c911c3b288908a99481df8c2830233ff4261352749e68908ceb2bb5"} Jul 14 10:01:21.193613 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:21.193583 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f9qfj" event={"ID":"6ba17703-e928-47bf-9518-f92ac981131e","Type":"ContainerStarted","Data":"96517885617fe00f475209b21b24f8251aa4fffaa54f0d888baea757a7cb9cd1"} Jul 14 10:01:22.263309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.263272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:22.263755 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.263343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:22.263755 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:22.263507 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 10:01:22.263755 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:22.263575 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls podName:8b0e354f-e5d2-4f1b-9324-2912327875b1 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:38.263557789 +0000 UTC m=+82.897621612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bwk7v" (UID: "8b0e354f-e5d2-4f1b-9324-2912327875b1") : secret "insights-runtime-extractor-tls" not found Jul 14 10:01:22.263984 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.263954 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b0e354f-e5d2-4f1b-9324-2912327875b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:22.668041 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.667956 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:22.668041 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.668027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:22.671612 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.671555 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"image-registry-9b4d7f958-n5tfw\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:22.672211 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.672131 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b9ad799-4001-448a-8947-36def65e3f94-metrics-tls\") pod \"dns-default-g72sn\" (UID: \"2b9ad799-4001-448a-8947-36def65e3f94\") " pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:22.911466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.911435 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zh9t2\"" Jul 14 10:01:22.919759 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.919633 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:22.925320 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.925290 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d25fg\"" Jul 14 10:01:22.933531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:22.933502 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:23.066476 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:23.066401 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:01:25.391972 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:25.391940 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb4c5da_0352_404b_9005_b9a2ea7fdf6d.slice/crio-77f96f4db04845c744f6e73726a173947742d612f3e7e42f446ea3b6e0e092f5 WatchSource:0}: Error finding container 77f96f4db04845c744f6e73726a173947742d612f3e7e42f446ea3b6e0e092f5: Status 404 returned error can't find the container with id 77f96f4db04845c744f6e73726a173947742d612f3e7e42f446ea3b6e0e092f5 Jul 14 10:01:25.857206 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:25.857178 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g72sn"] Jul 14 10:01:25.869122 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:25.869099 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9ad799_4001_448a_8947_36def65e3f94.slice/crio-7b05c286049804fb40115c121f291f15baf4e708026f964be727413a15b25333 WatchSource:0}: Error finding container 7b05c286049804fb40115c121f291f15baf4e708026f964be727413a15b25333: Status 404 returned error can't find the container with id 7b05c286049804fb40115c121f291f15baf4e708026f964be727413a15b25333 Jul 14 10:01:26.211564 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.211529 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7nmln" event={"ID":"b68475b9-1f2f-4ad9-875c-ea54c2400cc0","Type":"ContainerStarted","Data":"53f470743eb6116338c1261173433df1b57f8e86d5bfd144fddb0822b51542e2"} Jul 14 10:01:26.213106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.213080 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" event={"ID":"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d","Type":"ContainerStarted","Data":"92c51801c1a237a827f7eb844e6d24632d783e18f9559467d2d51d7c26fff039"} Jul 14 10:01:26.213256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.213124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" event={"ID":"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d","Type":"ContainerStarted","Data":"77f96f4db04845c744f6e73726a173947742d612f3e7e42f446ea3b6e0e092f5"} Jul 14 10:01:26.213256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.213146 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:26.214562 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.214532 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f9qfj" event={"ID":"6ba17703-e928-47bf-9518-f92ac981131e","Type":"ContainerStarted","Data":"b2ed6d4091ba0e9a6ba44cc703b50288f9262da48e10df004eb3c38741052eb2"} Jul 14 10:01:26.214720 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.214638 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:01:26.215650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.215627 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g72sn" event={"ID":"2b9ad799-4001-448a-8947-36def65e3f94","Type":"ContainerStarted","Data":"7b05c286049804fb40115c121f291f15baf4e708026f964be727413a15b25333"} Jul 14 10:01:26.227068 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.227020 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7nmln" podStartSLOduration=65.526489478 podStartE2EDuration="1m10.227007339s" totalImagesPullingTime="4.700517861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:26.226610127 +0000 UTC m=+70.860673968" watchObservedRunningTime="2026-07-14 10:01:26.227007339 +0000 UTC m=+70.861071201" Jul 14 10:01:26.240997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.240950 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f9qfj" podStartSLOduration=66.542806529 podStartE2EDuration="1m11.240935219s" totalImagesPullingTime="4.69812869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:26.240584718 +0000 UTC m=+70.874648558" watchObservedRunningTime="2026-07-14 10:01:26.240935219 +0000 UTC m=+70.874999062" Jul 14 10:01:26.258970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:26.258918 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" podStartSLOduration=50.258899368 podStartE2EDuration="50.258899368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:26.25777625 +0000 UTC m=+70.891840091" watchObservedRunningTime="2026-07-14 10:01:26.258899368 +0000 UTC m=+70.892963210" Jul 14 10:01:28.125329 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.125292 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf"] Jul 14 10:01:28.128009 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.127988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:28.130621 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.130594 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 10:01:28.130749 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.130592 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-b895p\"" Jul 14 10:01:28.135197 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.135173 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:01:28.137855 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.137832 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf"] Jul 14 10:01:28.212583 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.212551 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/81f21ca6-03a1-4204-8187-5979c5b36f11-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hvccf\" (UID: \"81f21ca6-03a1-4204-8187-5979c5b36f11\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:28.313477 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.313440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/81f21ca6-03a1-4204-8187-5979c5b36f11-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hvccf\" (UID: \"81f21ca6-03a1-4204-8187-5979c5b36f11\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:28.316280 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.316250 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/81f21ca6-03a1-4204-8187-5979c5b36f11-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-hvccf\" (UID: \"81f21ca6-03a1-4204-8187-5979c5b36f11\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:28.437454 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.437423 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:28.560167 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:28.560112 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf"] Jul 14 10:01:28.564593 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:28.564563 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81f21ca6_03a1_4204_8187_5979c5b36f11.slice/crio-0038f47e82b6892256aa7f94f052e13c05fb4469ec71e9363aa9c9f2f11a5385 WatchSource:0}: Error finding container 0038f47e82b6892256aa7f94f052e13c05fb4469ec71e9363aa9c9f2f11a5385: Status 404 returned error can't find the container with id 0038f47e82b6892256aa7f94f052e13c05fb4469ec71e9363aa9c9f2f11a5385 Jul 14 10:01:29.225654 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:29.225626 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g72sn" event={"ID":"2b9ad799-4001-448a-8947-36def65e3f94","Type":"ContainerStarted","Data":"c717b77b4ca7721471ed6423cec4d2e7566f0bfc3d73aa18b6601900c926620b"} Jul 14 10:01:29.226633 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:29.226611 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" event={"ID":"81f21ca6-03a1-4204-8187-5979c5b36f11","Type":"ContainerStarted","Data":"0038f47e82b6892256aa7f94f052e13c05fb4469ec71e9363aa9c9f2f11a5385"} Jul 14 10:01:30.232314 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.232279 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g72sn" event={"ID":"2b9ad799-4001-448a-8947-36def65e3f94","Type":"ContainerStarted","Data":"a0aae4f813a60fbbfd58ebee2d353c2767f3b59e56c07b468178c51c877776f1"} Jul 14 10:01:30.232755 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.232418 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:30.233676 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.233657 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" event={"ID":"81f21ca6-03a1-4204-8187-5979c5b36f11","Type":"ContainerStarted","Data":"1152c2003ae94302cb88f0b0623046a6a2131b3492297efc50c230caf3324352"} Jul 14 10:01:30.233758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.233696 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:30.238426 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.238402 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" Jul 14 10:01:30.248001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.247951 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g72sn" podStartSLOduration=37.013389925 podStartE2EDuration="40.247939552s" totalImagesPullingTime="3.234549627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:30.247681506 +0000 UTC m=+74.881745341" watchObservedRunningTime="2026-07-14 10:01:30.247939552 +0000 UTC m=+74.882003393" Jul 14 10:01:30.262133 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.262078 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-hvccf" podStartSLOduration=1.061579116 podStartE2EDuration="2.262064322s" totalImagesPullingTime="1.200485206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:30.261537074 +0000 UTC m=+74.895600916" watchObservedRunningTime="2026-07-14 10:01:30.262064322 +0000 UTC m=+74.896128162" Jul 14 10:01:30.920785 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.920744 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-xdd5t"] Jul 14 10:01:30.922984 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.922960 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:30.925455 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.925431 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vvrnm\"" Jul 14 10:01:30.925560 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.925479 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 10:01:30.926758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.926730 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 10:01:30.926846 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.926730 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 10:01:30.926919 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.926892 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 10:01:30.926970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.926933 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 10:01:30.933648 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:30.933623 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-xdd5t"] Jul 14 10:01:31.034241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.034204 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018133ad-1ded-49af-9302-5f5a9810e78d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.034412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.034251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.034412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.034291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.034412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.034309 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blmgv\" (UniqueName: \"kubernetes.io/projected/018133ad-1ded-49af-9302-5f5a9810e78d-kube-api-access-blmgv\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.134769 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.134744 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.134904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.134790 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.134904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.134809 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blmgv\" (UniqueName: \"kubernetes.io/projected/018133ad-1ded-49af-9302-5f5a9810e78d-kube-api-access-blmgv\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.134904 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:31.134892 2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 14 10:01:31.135007 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:31.134949 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls podName:018133ad-1ded-49af-9302-5f5a9810e78d nodeName:}" failed. No retries permitted until 2026-07-14 10:01:31.634933768 +0000 UTC m=+76.268997590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls") pod "prometheus-operator-64785b6675-xdd5t" (UID: "018133ad-1ded-49af-9302-5f5a9810e78d") : secret "prometheus-operator-tls" not found Jul 14 10:01:31.135051 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.135018 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018133ad-1ded-49af-9302-5f5a9810e78d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.135630 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.135612 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/018133ad-1ded-49af-9302-5f5a9810e78d-metrics-client-ca\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.137066 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.137051 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.143332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.143309 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blmgv\" (UniqueName: \"kubernetes.io/projected/018133ad-1ded-49af-9302-5f5a9810e78d-kube-api-access-blmgv\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.326849 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.326814 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:01:31.329922 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.329905 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.332675 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332650 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 10:01:31.332788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332736 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 10:01:31.332880 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332862 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5ltrq\"" Jul 14 10:01:31.332953 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332935 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 10:01:31.333000 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332986 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 10:01:31.333050 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.332986 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 10:01:31.333300 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.333285 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 10:01:31.333543 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.333529 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 10:01:31.338682 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.338661 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:01:31.436604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436566 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.436771 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436627 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.436771 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.436771 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9ms\" (UniqueName: \"kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.436875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436798 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.436875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.436825 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538115 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538067 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538137 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538173 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538190 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9ms\" (UniqueName: \"kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538397 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538397 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538313 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538860 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538837 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538951 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538887 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.538951 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.538910 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.540543 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.540521 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.540715 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.540700 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.546282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.546264 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9ms\" (UniqueName: \"kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms\") pod \"console-5967b6fd8-p9qt5\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.639321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.639225 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:31.639463 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.639361 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.641595 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.641573 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/018133ad-1ded-49af-9302-5f5a9810e78d-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-xdd5t\" (UID: \"018133ad-1ded-49af-9302-5f5a9810e78d\") " pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.763474 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.763413 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:01:31.766010 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:31.765982 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb475dbf_76e6_490c_af84_0a7e50fa3d21.slice/crio-1d559f75d2e2b619fadd38d15d2d683d42b3b19a02065f40b7de920cebe44922 WatchSource:0}: Error finding container 1d559f75d2e2b619fadd38d15d2d683d42b3b19a02065f40b7de920cebe44922: Status 404 returned error can't find the container with id 1d559f75d2e2b619fadd38d15d2d683d42b3b19a02065f40b7de920cebe44922 Jul 14 10:01:31.831957 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.831914 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" Jul 14 10:01:31.955120 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:31.955078 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-xdd5t"] Jul 14 10:01:31.958657 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:31.958633 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018133ad_1ded_49af_9302_5f5a9810e78d.slice/crio-1f12fefbff8d8ede0be3e1dd252383fe788c2606a672bab0f57a8681b542eff9 WatchSource:0}: Error finding container 1f12fefbff8d8ede0be3e1dd252383fe788c2606a672bab0f57a8681b542eff9: Status 404 returned error can't find the container with id 1f12fefbff8d8ede0be3e1dd252383fe788c2606a672bab0f57a8681b542eff9 Jul 14 10:01:32.241720 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:32.241684 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5967b6fd8-p9qt5" event={"ID":"fb475dbf-76e6-490c-af84-0a7e50fa3d21","Type":"ContainerStarted","Data":"1d559f75d2e2b619fadd38d15d2d683d42b3b19a02065f40b7de920cebe44922"} Jul 14 10:01:32.246413 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:32.246384 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" event={"ID":"018133ad-1ded-49af-9302-5f5a9810e78d","Type":"ContainerStarted","Data":"1f12fefbff8d8ede0be3e1dd252383fe788c2606a672bab0f57a8681b542eff9"} Jul 14 10:01:36.261127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:36.261086 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" event={"ID":"018133ad-1ded-49af-9302-5f5a9810e78d","Type":"ContainerStarted","Data":"832e321f9e3debdb3e88f84ba66d6d58146a5e8732620a0150923c48cbfe6478"} Jul 14 10:01:36.261677 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:36.261165 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" event={"ID":"018133ad-1ded-49af-9302-5f5a9810e78d","Type":"ContainerStarted","Data":"dc63640cba5edc193244a38aa0b6d29bd276b53dc8b4b6f947eea6764a94746f"} Jul 14 10:01:36.262627 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:36.262606 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5967b6fd8-p9qt5" event={"ID":"fb475dbf-76e6-490c-af84-0a7e50fa3d21","Type":"ContainerStarted","Data":"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c"} Jul 14 10:01:36.277608 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:36.277552 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-xdd5t" podStartSLOduration=2.773399695 podStartE2EDuration="6.277535915s" totalImagesPullingTime="3.50413622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:36.276963504 +0000 UTC m=+80.911027356" watchObservedRunningTime="2026-07-14 10:01:36.277535915 +0000 UTC m=+80.911599758" Jul 14 10:01:36.292646 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:36.292598 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5967b6fd8-p9qt5" podStartSLOduration=1.59369942 podStartE2EDuration="5.292582337s" totalImagesPullingTime="3.698882917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:36.291634047 +0000 UTC m=+80.925697888" watchObservedRunningTime="2026-07-14 10:01:36.292582337 +0000 UTC m=+80.926646178" Jul 14 10:01:37.188552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:37.188408 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:37.190803 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:37.190771 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d8da39dc-4ed7-4fd9-8221-6bc169ecb783-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vpnjq\" (UID: \"d8da39dc-4ed7-4fd9-8221-6bc169ecb783\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:37.440462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:37.440379 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tn8fb\"" Jul 14 10:01:37.448420 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:37.448391 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" Jul 14 10:01:37.591778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:37.591742 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vpnjq"] Jul 14 10:01:37.595299 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:37.595263 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8da39dc_4ed7_4fd9_8221_6bc169ecb783.slice/crio-84d8dc234b57f773dcbf233cf484edd4a3f7c5fdffc95ac90887dc42bc4f1daf WatchSource:0}: Error finding container 84d8dc234b57f773dcbf233cf484edd4a3f7c5fdffc95ac90887dc42bc4f1daf: Status 404 returned error can't find the container with id 84d8dc234b57f773dcbf233cf484edd4a3f7c5fdffc95ac90887dc42bc4f1daf Jul 14 10:01:38.249023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.248989 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z"] Jul 14 10:01:38.252318 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.252301 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.254990 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.254965 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 10:01:38.255116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.255079 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kztvb\"" Jul 14 10:01:38.255191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.255173 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 10:01:38.269629 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.269592 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" event={"ID":"d8da39dc-4ed7-4fd9-8221-6bc169ecb783","Type":"ContainerStarted","Data":"84d8dc234b57f773dcbf233cf484edd4a3f7c5fdffc95ac90887dc42bc4f1daf"} Jul 14 10:01:38.275883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.275860 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z"] Jul 14 10:01:38.298253 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.298226 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:38.301239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.301212 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b0e354f-e5d2-4f1b-9324-2912327875b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bwk7v\" (UID: \"8b0e354f-e5d2-4f1b-9324-2912327875b1\") " pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:38.302396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.302377 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rc45b"] Jul 14 10:01:38.304142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.304128 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-szrgf\"" Jul 14 10:01:38.307005 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.306987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.310304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.310281 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 10:01:38.310304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.310281 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 10:01:38.310519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.310506 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 10:01:38.311224 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.311208 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wkhrf\"" Jul 14 10:01:38.311699 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.311683 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bwk7v" Jul 14 10:01:38.399391 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399354 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399404 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-textfile\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399439 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399474 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-sys\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399498 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-tls\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-root\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74kj2\" (UniqueName: \"kubernetes.io/projected/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-kube-api-access-74kj2\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399588 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399617 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399644 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399668 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-wtmp\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399692 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzgng\" (UniqueName: \"kubernetes.io/projected/45c7857b-1cf4-4805-9f23-a41bf4355042-kube-api-access-wzgng\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.399828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.399712 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-metrics-client-ca\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.460229 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.460195 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bwk7v"] Jul 14 10:01:38.464651 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:38.464619 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0e354f_e5d2_4f1b_9324_2912327875b1.slice/crio-2d1095448b6770815552a1830681a748127ff4fe0c23df31e792c7744fa88240 WatchSource:0}: Error finding container 2d1095448b6770815552a1830681a748127ff4fe0c23df31e792c7744fa88240: Status 404 returned error can't find the container with id 2d1095448b6770815552a1830681a748127ff4fe0c23df31e792c7744fa88240 Jul 14 10:01:38.500667 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-metrics-client-ca\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.500667 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500653 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500686 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-textfile\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500715 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500750 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-sys\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500772 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-tls\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-root\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500822 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74kj2\" (UniqueName: \"kubernetes.io/projected/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-kube-api-access-74kj2\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.500870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500858 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500888 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.501267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500914 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500938 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-wtmp\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.500964 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzgng\" (UniqueName: \"kubernetes.io/projected/45c7857b-1cf4-4805-9f23-a41bf4355042-kube-api-access-wzgng\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501457 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.501371 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-root\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501497 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.501476 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-textfile\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501545 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.501525 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-wtmp\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501579 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.501546 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45c7857b-1cf4-4805-9f23-a41bf4355042-sys\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.501904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.501856 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.502266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.502240 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-metrics-client-ca\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.503004 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.502701 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-accelerators-collector-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.504330 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.504305 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.504633 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.504609 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.504675 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.504609 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.504675 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.504659 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/45c7857b-1cf4-4805-9f23-a41bf4355042-node-exporter-tls\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.510638 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.510614 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzgng\" (UniqueName: \"kubernetes.io/projected/45c7857b-1cf4-4805-9f23-a41bf4355042-kube-api-access-wzgng\") pod \"node-exporter-rc45b\" (UID: \"45c7857b-1cf4-4805-9f23-a41bf4355042\") " pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.510846 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.510829 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74kj2\" (UniqueName: \"kubernetes.io/projected/90eccaaf-0c5e-4e60-8b5b-846e45979f4e-kube-api-access-74kj2\") pod \"openshift-state-metrics-84bd68b896-4mg8z\" (UID: \"90eccaaf-0c5e-4e60-8b5b-846e45979f4e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.561865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.561827 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" Jul 14 10:01:38.616269 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.615971 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rc45b" Jul 14 10:01:38.629850 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:38.629809 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c7857b_1cf4_4805_9f23_a41bf4355042.slice/crio-a5f4fc9ab16256891ce1f1db726d0b20a0d0db7cf84532d260b4f7838dff869f WatchSource:0}: Error finding container a5f4fc9ab16256891ce1f1db726d0b20a0d0db7cf84532d260b4f7838dff869f: Status 404 returned error can't find the container with id a5f4fc9ab16256891ce1f1db726d0b20a0d0db7cf84532d260b4f7838dff869f Jul 14 10:01:38.707043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:38.707000 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z"] Jul 14 10:01:38.942321 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:38.942288 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90eccaaf_0c5e_4e60_8b5b_846e45979f4e.slice/crio-9a088c7624a33c8b8b207b226c36729284b4402be204a19b90049428963d1d13 WatchSource:0}: Error finding container 9a088c7624a33c8b8b207b226c36729284b4402be204a19b90049428963d1d13: Status 404 returned error can't find the container with id 9a088c7624a33c8b8b207b226c36729284b4402be204a19b90049428963d1d13 Jul 14 10:01:39.274462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.274368 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" event={"ID":"d8da39dc-4ed7-4fd9-8221-6bc169ecb783","Type":"ContainerStarted","Data":"64ed24da56eb77fb1e53a8d36ceaa4108c4bc6f8cb0ec0b7740dd30bafbcddc6"} Jul 14 10:01:39.275624 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.275588 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwk7v" event={"ID":"8b0e354f-e5d2-4f1b-9324-2912327875b1","Type":"ContainerStarted","Data":"2d1095448b6770815552a1830681a748127ff4fe0c23df31e792c7744fa88240"} Jul 14 10:01:39.278106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.277637 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" event={"ID":"90eccaaf-0c5e-4e60-8b5b-846e45979f4e","Type":"ContainerStarted","Data":"778ddbfc77f7236178c022bb7fed1a8c64da69951af2a9a4dce9550f658efc95"} Jul 14 10:01:39.278106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.277690 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" event={"ID":"90eccaaf-0c5e-4e60-8b5b-846e45979f4e","Type":"ContainerStarted","Data":"37358ba8ff083309b47b3661f2b04d524a6f94836c70fd9fdaa78fd0a33801fc"} Jul 14 10:01:39.278106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.277718 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" event={"ID":"90eccaaf-0c5e-4e60-8b5b-846e45979f4e","Type":"ContainerStarted","Data":"9a088c7624a33c8b8b207b226c36729284b4402be204a19b90049428963d1d13"} Jul 14 10:01:39.279542 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.279516 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc45b" event={"ID":"45c7857b-1cf4-4805-9f23-a41bf4355042","Type":"ContainerStarted","Data":"a5f4fc9ab16256891ce1f1db726d0b20a0d0db7cf84532d260b4f7838dff869f"} Jul 14 10:01:39.292668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:39.292613 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-vpnjq" podStartSLOduration=32.918741539 podStartE2EDuration="34.292596162s" totalImagesPullingTime="1.373854623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:39.292120932 +0000 UTC m=+83.926184775" watchObservedRunningTime="2026-07-14 10:01:39.292596162 +0000 UTC m=+83.926660004" Jul 14 10:01:40.239434 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.239401 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g72sn" Jul 14 10:01:40.255950 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.255919 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5cd48bf654-hztls"] Jul 14 10:01:40.258829 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.258803 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.261509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.261446 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 10:01:40.262170 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262114 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 10:01:40.262326 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262206 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 10:01:40.262326 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262289 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 10:01:40.262556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262329 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 10:01:40.262556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262347 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1ovcfh2k58soh\"" Jul 14 10:01:40.262745 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.262726 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-r8m9b\"" Jul 14 10:01:40.275773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.275718 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cd48bf654-hztls"] Jul 14 10:01:40.418035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.417918 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.417966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.417997 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.418061 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9d9f49-f235-4942-9ba2-fd59b5396f74-metrics-client-ca\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.418130 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.418190 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-grpc-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.418220 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrt6\" (UniqueName: \"kubernetes.io/projected/3a9d9f49-f235-4942-9ba2-fd59b5396f74-kube-api-access-twrt6\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.418358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.418248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519337 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519255 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-grpc-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519337 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519297 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrt6\" (UniqueName: \"kubernetes.io/projected/3a9d9f49-f235-4942-9ba2-fd59b5396f74-kube-api-access-twrt6\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519337 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519321 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519419 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519479 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519503 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9d9f49-f235-4942-9ba2-fd59b5396f74-metrics-client-ca\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.519611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.519552 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.520436 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.520382 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9d9f49-f235-4942-9ba2-fd59b5396f74-metrics-client-ca\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.522850 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.522823 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-grpc-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.523336 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.523278 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-tls\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.523442 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.523345 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.524104 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.523706 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.524104 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.524070 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.525669 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.525611 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9d9f49-f235-4942-9ba2-fd59b5396f74-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.528364 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.528345 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrt6\" (UniqueName: \"kubernetes.io/projected/3a9d9f49-f235-4942-9ba2-fd59b5396f74-kube-api-access-twrt6\") pod \"thanos-querier-5cd48bf654-hztls\" (UID: \"3a9d9f49-f235-4942-9ba2-fd59b5396f74\") " pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:40.572251 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:40.572212 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:41.639903 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:41.639867 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:41.640396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:41.639934 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:41.645517 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:41.645489 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:42.295864 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.295833 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:01:42.587818 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.587747 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-55bcfc6b77-56pxm"] Jul 14 10:01:42.596988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.596962 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.599528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.599467 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kzgfg\"" Jul 14 10:01:42.599528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.599499 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3gdvvmbdk0s8h\"" Jul 14 10:01:42.599923 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.599900 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55bcfc6b77-56pxm"] Jul 14 10:01:42.600693 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.600662 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 10:01:42.600881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.600863 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 10:01:42.601228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.601211 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 10:01:42.601228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.601228 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 10:01:42.740485 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740447 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-metrics-server-audit-profiles\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740512 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-client-certs\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740567 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-tls\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740594 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-client-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740632 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4jpx\" (UniqueName: \"kubernetes.io/projected/676119f7-ccf9-42dd-a463-c70636c521d3-kube-api-access-h4jpx\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/676119f7-ccf9-42dd-a463-c70636c521d3-audit-log\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.740893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.740746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.841812 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841717 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/676119f7-ccf9-42dd-a463-c70636c521d3-audit-log\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.841812 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841767 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841816 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-metrics-server-audit-profiles\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841859 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-client-certs\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841903 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-tls\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841924 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-client-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.841947 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4jpx\" (UniqueName: \"kubernetes.io/projected/676119f7-ccf9-42dd-a463-c70636c521d3-kube-api-access-h4jpx\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842316 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.842209 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/676119f7-ccf9-42dd-a463-c70636c521d3-audit-log\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.842723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.842698 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.843034 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.843003 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/676119f7-ccf9-42dd-a463-c70636c521d3-metrics-server-audit-profiles\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.844861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.844834 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-tls\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.844961 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.844842 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-client-ca-bundle\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.844961 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.844927 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/676119f7-ccf9-42dd-a463-c70636c521d3-secret-metrics-server-client-certs\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.850241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.850145 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4jpx\" (UniqueName: \"kubernetes.io/projected/676119f7-ccf9-42dd-a463-c70636c521d3-kube-api-access-h4jpx\") pod \"metrics-server-55bcfc6b77-56pxm\" (UID: \"676119f7-ccf9-42dd-a463-c70636c521d3\") " pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:42.908299 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:42.908260 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:01:43.043212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.043146 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq"] Jul 14 10:01:43.045793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.045765 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.048469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.048343 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pcg4k\"" Jul 14 10:01:43.048469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.048399 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 14 10:01:43.057475 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.057435 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq"] Jul 14 10:01:43.143971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.143941 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.144103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.144061 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fzf9\" (UniqueName: \"kubernetes.io/projected/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-kube-api-access-7fzf9\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.244834 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.244519 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.244834 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.244606 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fzf9\" (UniqueName: \"kubernetes.io/projected/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-kube-api-access-7fzf9\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.244834 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:43.244695 2569 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 14 10:01:43.244834 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:01:43.244753 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert podName:44a6f46b-05a0-46f8-8015-efc5b5bf31f0 nodeName:}" failed. No retries permitted until 2026-07-14 10:01:43.744733927 +0000 UTC m=+88.378797753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-nm5tq" (UID: "44a6f46b-05a0-46f8-8015-efc5b5bf31f0") : secret "monitoring-plugin-cert" not found Jul 14 10:01:43.256226 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.256134 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fzf9\" (UniqueName: \"kubernetes.io/projected/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-kube-api-access-7fzf9\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.290894 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.290866 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cd48bf654-hztls"] Jul 14 10:01:43.293747 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:43.293685 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9d9f49_f235_4942_9ba2_fd59b5396f74.slice/crio-84d87b304aa7dd04ff366f0b84bfe6c4ec857d705b741002c1da430f5e3fce7e WatchSource:0}: Error finding container 84d87b304aa7dd04ff366f0b84bfe6c4ec857d705b741002c1da430f5e3fce7e: Status 404 returned error can't find the container with id 84d87b304aa7dd04ff366f0b84bfe6c4ec857d705b741002c1da430f5e3fce7e Jul 14 10:01:43.297708 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.297683 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwk7v" event={"ID":"8b0e354f-e5d2-4f1b-9324-2912327875b1","Type":"ContainerStarted","Data":"fbf9ff24fdf87623814b48176539532291f156bb24f7ff93fe2d281d715436c5"} Jul 14 10:01:43.300173 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.300118 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" event={"ID":"90eccaaf-0c5e-4e60-8b5b-846e45979f4e","Type":"ContainerStarted","Data":"0cb4eb9768ff2f568c8059f6268e350ed620dab2da85bddc86e49f70f5ae6056"} Jul 14 10:01:43.301752 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.301732 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc45b" event={"ID":"45c7857b-1cf4-4805-9f23-a41bf4355042","Type":"ContainerStarted","Data":"61f1d069f4a576bbbb1c887b49b931c84b4d2dc33aefe84aa20ac220ee5752d5"} Jul 14 10:01:43.307239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.307192 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55bcfc6b77-56pxm"] Jul 14 10:01:43.310640 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:43.310611 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676119f7_ccf9_42dd_a463_c70636c521d3.slice/crio-21fcbf6cb467fbdc86d528fab35e25b437be414a859acefa1255da1b430f4b11 WatchSource:0}: Error finding container 21fcbf6cb467fbdc86d528fab35e25b437be414a859acefa1255da1b430f4b11: Status 404 returned error can't find the container with id 21fcbf6cb467fbdc86d528fab35e25b437be414a859acefa1255da1b430f4b11 Jul 14 10:01:43.319394 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.319346 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4mg8z" podStartSLOduration=1.2286889269999999 podStartE2EDuration="5.319330389s" totalImagesPullingTime="4.090641462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:43.318295141 +0000 UTC m=+87.952358982" watchObservedRunningTime="2026-07-14 10:01:43.319330389 +0000 UTC m=+87.953394227" Jul 14 10:01:43.749515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.749488 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.751795 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.751773 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/44a6f46b-05a0-46f8-8015-efc5b5bf31f0-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-nm5tq\" (UID: \"44a6f46b-05a0-46f8-8015-efc5b5bf31f0\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:43.956561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:43.956527 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:44.104085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.104046 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq"] Jul 14 10:01:44.107749 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:44.107718 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a6f46b_05a0_46f8_8015_efc5b5bf31f0.slice/crio-4aa168c9c5d418a8a6549ad2881de3a242deb3df71f31a108cd13fd0c47ec664 WatchSource:0}: Error finding container 4aa168c9c5d418a8a6549ad2881de3a242deb3df71f31a108cd13fd0c47ec664: Status 404 returned error can't find the container with id 4aa168c9c5d418a8a6549ad2881de3a242deb3df71f31a108cd13fd0c47ec664 Jul 14 10:01:44.311843 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.311746 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwk7v" event={"ID":"8b0e354f-e5d2-4f1b-9324-2912327875b1","Type":"ContainerStarted","Data":"ef809b38a26810483be9291e39db98b672b6f84a3a00deecd920f4469be88d56"} Jul 14 10:01:44.313400 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.313358 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" event={"ID":"44a6f46b-05a0-46f8-8015-efc5b5bf31f0","Type":"ContainerStarted","Data":"4aa168c9c5d418a8a6549ad2881de3a242deb3df71f31a108cd13fd0c47ec664"} Jul 14 10:01:44.314884 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.314850 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" event={"ID":"676119f7-ccf9-42dd-a463-c70636c521d3","Type":"ContainerStarted","Data":"21fcbf6cb467fbdc86d528fab35e25b437be414a859acefa1255da1b430f4b11"} Jul 14 10:01:44.316801 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.316760 2569 generic.go:356] "Generic (PLEG): container finished" podID="45c7857b-1cf4-4805-9f23-a41bf4355042" containerID="61f1d069f4a576bbbb1c887b49b931c84b4d2dc33aefe84aa20ac220ee5752d5" exitCode=0 Jul 14 10:01:44.316932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.316838 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc45b" event={"ID":"45c7857b-1cf4-4805-9f23-a41bf4355042","Type":"ContainerDied","Data":"61f1d069f4a576bbbb1c887b49b931c84b4d2dc33aefe84aa20ac220ee5752d5"} Jul 14 10:01:44.320775 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.320727 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"84d87b304aa7dd04ff366f0b84bfe6c4ec857d705b741002c1da430f5e3fce7e"} Jul 14 10:01:44.659971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:44.659891 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:01:45.328328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:45.328280 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc45b" event={"ID":"45c7857b-1cf4-4805-9f23-a41bf4355042","Type":"ContainerStarted","Data":"879e273b65fcf6280360fbdd0a50f02ab48bba1a6482387877fcf0e1d72ce276"} Jul 14 10:01:45.328792 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:45.328353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rc45b" event={"ID":"45c7857b-1cf4-4805-9f23-a41bf4355042","Type":"ContainerStarted","Data":"385c8fa2d832a1131797e54e5ec2e51db5c301dce296a84f74216b58c4db4616"} Jul 14 10:01:45.350053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:45.349994 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rc45b" podStartSLOduration=2.755833468 podStartE2EDuration="7.349975485s" totalImagesPullingTime="4.50559817s" totalInitContainerRuntime="88.543847ms" isStatefulPod=true podCreationTimestamp="2026-07-14 10:01:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:45.34795376 +0000 UTC m=+89.982017613" watchObservedRunningTime="2026-07-14 10:01:45.349975485 +0000 UTC m=+89.984039325" Jul 14 10:01:47.340707 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.340669 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bwk7v" event={"ID":"8b0e354f-e5d2-4f1b-9324-2912327875b1","Type":"ContainerStarted","Data":"8fb9aadc470d50e4f19c7788dda2b9fca6b20ffdff8f258afbb492597ad5c362"} Jul 14 10:01:47.342773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.342738 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" event={"ID":"44a6f46b-05a0-46f8-8015-efc5b5bf31f0","Type":"ContainerStarted","Data":"796cf259147ded95bbb82860c26615d462eec3fe7064516c72c757409e6e4427"} Jul 14 10:01:47.342909 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.342890 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:47.344366 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.344339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" event={"ID":"676119f7-ccf9-42dd-a463-c70636c521d3","Type":"ContainerStarted","Data":"4bcfee6aec63202cac1db8513320e91f4d3144463dbaea678acc4939209856a3"} Jul 14 10:01:47.352296 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.348932 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" Jul 14 10:01:47.369618 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.369565 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bwk7v" podStartSLOduration=33.222802997 podStartE2EDuration="41.369548169s" totalImagesPullingTime="8.146745172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:01:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:47.368846174 +0000 UTC m=+92.002910054" watchObservedRunningTime="2026-07-14 10:01:47.369548169 +0000 UTC m=+92.003612010" Jul 14 10:01:47.385137 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.385082 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" podStartSLOduration=3.520462079 podStartE2EDuration="5.385063539s" totalImagesPullingTime="1.86460146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:47.384913975 +0000 UTC m=+92.018977843" watchObservedRunningTime="2026-07-14 10:01:47.385063539 +0000 UTC m=+92.019127381" Jul 14 10:01:47.400262 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:47.400209 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-nm5tq" podStartSLOduration=1.781812705 podStartE2EDuration="4.400193044s" totalImagesPullingTime="2.618380339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:47.399774701 +0000 UTC m=+92.033838542" watchObservedRunningTime="2026-07-14 10:01:47.400193044 +0000 UTC m=+92.034256876" Jul 14 10:01:48.226530 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:48.226499 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:49.320491 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.320456 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:01:49.322864 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.322841 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.332388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.332360 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 10:01:49.333469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.333447 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:01:49.355691 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.355662 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"59356ac6b84e24cfaab4f5699a5435601e1b415ee0be435b95927d7c28c582d9"} Jul 14 10:01:49.355825 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.355726 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"a6bb6cd475250394ffbe7b30490200eb06995ae19a2de763472c06b8764ca74e"} Jul 14 10:01:49.355825 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.355762 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"e54337241fd832dc9841547cdb28621e70b83c5e06e7a3aa93e489f16d47cff2"} Jul 14 10:01:49.504023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.503985 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504050 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504175 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504209 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpzwk\" (UniqueName: \"kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.504282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.504278 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.604794 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604710 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.604794 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604752 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzwk\" (UniqueName: \"kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.604794 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604781 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604803 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604867 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.604884 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.605534 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.605598 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.605620 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.605847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.605823 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.607395 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.607377 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.607474 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.607449 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.613087 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.613055 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpzwk\" (UniqueName: \"kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk\") pod \"console-7978795d7-tvbtc\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.631985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.631952 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:49.760827 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:49.760794 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:01:49.766716 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:01:49.766692 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e412b5c_e8cd_47e6_b73a_3662e8a3b190.slice/crio-d06b7949e016db397821d2a2e9361c92b0131d5b8b4ac81d0407ba80c8232932 WatchSource:0}: Error finding container d06b7949e016db397821d2a2e9361c92b0131d5b8b4ac81d0407ba80c8232932: Status 404 returned error can't find the container with id d06b7949e016db397821d2a2e9361c92b0131d5b8b4ac81d0407ba80c8232932 Jul 14 10:01:50.364098 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.364063 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"51a96fffb1a4bac88c7c9ddf7010ed22b283b61237102aff047b3a2086c271d7"} Jul 14 10:01:50.364531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.364127 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"dd5c2e9c0911bf2b960563e79acbb4405ed9acd340cee84287eebc044a9e5b02"} Jul 14 10:01:50.364531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.364182 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" event={"ID":"3a9d9f49-f235-4942-9ba2-fd59b5396f74","Type":"ContainerStarted","Data":"9f4c3c8c6ead73fddad38f6903b01f0e6a643c651abf1e49702094c908c6f944"} Jul 14 10:01:50.364531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.364337 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:50.365837 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.365814 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978795d7-tvbtc" event={"ID":"5e412b5c-e8cd-47e6-b73a-3662e8a3b190","Type":"ContainerStarted","Data":"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf"} Jul 14 10:01:50.365918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.365852 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978795d7-tvbtc" event={"ID":"5e412b5c-e8cd-47e6-b73a-3662e8a3b190","Type":"ContainerStarted","Data":"d06b7949e016db397821d2a2e9361c92b0131d5b8b4ac81d0407ba80c8232932"} Jul 14 10:01:50.384461 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.384412 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" podStartSLOduration=3.694903019 podStartE2EDuration="10.384395631s" totalImagesPullingTime="6.689492612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:50.383787659 +0000 UTC m=+95.017851539" watchObservedRunningTime="2026-07-14 10:01:50.384395631 +0000 UTC m=+95.018459471" Jul 14 10:01:50.402038 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:50.401988 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7978795d7-tvbtc" podStartSLOduration=1.401971208 podStartE2EDuration="1.401971208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:01:50.401564222 +0000 UTC m=+95.035628065" watchObservedRunningTime="2026-07-14 10:01:50.401971208 +0000 UTC m=+95.036035048" Jul 14 10:01:53.240974 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.240908 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" podUID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" containerName="registry" containerID="cri-o://92c51801c1a237a827f7eb844e6d24632d783e18f9559467d2d51d7c26fff039" gracePeriod=30 Jul 14 10:01:53.380585 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.380550 2569 generic.go:356] "Generic (PLEG): container finished" podID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" containerID="92c51801c1a237a827f7eb844e6d24632d783e18f9559467d2d51d7c26fff039" exitCode=0 Jul 14 10:01:53.380734 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.380590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" event={"ID":"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d","Type":"ContainerDied","Data":"92c51801c1a237a827f7eb844e6d24632d783e18f9559467d2d51d7c26fff039"} Jul 14 10:01:53.484536 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.484509 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:53.540402 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540315 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540402 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540354 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540404 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540438 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540465 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwxfp\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540506 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540531 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540573 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets\") pod \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\" (UID: \"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d\") " Jul 14 10:01:53.540979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.540802 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:01:53.541789 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.541755 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:01:53.543248 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.543207 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:01:53.543349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.543250 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:01:53.543349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.543320 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp" (OuterVolumeSpecName: "kube-api-access-lwxfp") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "kube-api-access-lwxfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:01:53.543423 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.543343 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:01:53.543587 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.543569 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:01:53.549683 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.549656 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" (UID: "bcb4c5da-0352-404b-9005-b9a2ea7fdf6d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.641931 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-bound-sa-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.641965 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-ca-trust-extracted\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.641980 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-image-registry-private-configuration\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.641994 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwxfp\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-kube-api-access-lwxfp\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.642008 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-certificates\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.641967 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.642020 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-registry-tls\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.642403 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.642032 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-installation-pull-secrets\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:53.642403 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:53.642044 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d-trusted-ca\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:01:54.386078 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:54.386047 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" Jul 14 10:01:54.386552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:54.386046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9b4d7f958-n5tfw" event={"ID":"bcb4c5da-0352-404b-9005-b9a2ea7fdf6d","Type":"ContainerDied","Data":"77f96f4db04845c744f6e73726a173947742d612f3e7e42f446ea3b6e0e092f5"} Jul 14 10:01:54.386552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:54.386213 2569 scope.go:117] "RemoveContainer" containerID="92c51801c1a237a827f7eb844e6d24632d783e18f9559467d2d51d7c26fff039" Jul 14 10:01:54.403339 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:54.403310 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:01:54.406584 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:54.406557 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9b4d7f958-n5tfw"] Jul 14 10:01:55.869381 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:55.869347 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" path="/var/lib/kubelet/pods/bcb4c5da-0352-404b-9005-b9a2ea7fdf6d/volumes" Jul 14 10:01:56.376633 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:56.376607 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5cd48bf654-hztls" Jul 14 10:01:57.220515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:57.220482 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f9qfj" Jul 14 10:01:59.632900 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:59.632846 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:59.633310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:59.632920 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:01:59.637687 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:01:59.637646 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:02:00.416881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:00.416851 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:02:02.908981 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:02.908946 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:02:02.909402 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:02.909027 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:02:09.699375 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:09.699325 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5967b6fd8-p9qt5" podUID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" containerName="console" containerID="cri-o://8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c" gracePeriod=15 Jul 14 10:02:10.002088 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.002058 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:02:10.076111 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076079 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076306 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076127 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076306 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076169 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk9ms\" (UniqueName: \"kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076306 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076214 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076306 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076254 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076306 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076283 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert\") pod \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\" (UID: \"fb475dbf-76e6-490c-af84-0a7e50fa3d21\") " Jul 14 10:02:10.076625 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076595 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:02:10.076757 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076689 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:02:10.076836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.076777 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config" (OuterVolumeSpecName: "console-config") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:02:10.078590 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.078561 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:02:10.078590 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.078562 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms" (OuterVolumeSpecName: "kube-api-access-zk9ms") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "kube-api-access-zk9ms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:02:10.078705 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.078639 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb475dbf-76e6-490c-af84-0a7e50fa3d21" (UID: "fb475dbf-76e6-490c-af84-0a7e50fa3d21"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:02:10.177555 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177521 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-oauth-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.177555 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177550 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-oauth-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.177555 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177559 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk9ms\" (UniqueName: \"kubernetes.io/projected/fb475dbf-76e6-490c-af84-0a7e50fa3d21-kube-api-access-zk9ms\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.177789 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177569 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.177789 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177577 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb475dbf-76e6-490c-af84-0a7e50fa3d21-service-ca\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.177789 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.177586 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb475dbf-76e6-490c-af84-0a7e50fa3d21-console-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:02:10.453277 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.453247 2569 generic.go:356] "Generic (PLEG): container finished" podID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" containerID="8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c" exitCode=0 Jul 14 10:02:10.453487 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.453306 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5967b6fd8-p9qt5" Jul 14 10:02:10.453487 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.453339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5967b6fd8-p9qt5" event={"ID":"fb475dbf-76e6-490c-af84-0a7e50fa3d21","Type":"ContainerDied","Data":"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c"} Jul 14 10:02:10.453487 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.453387 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5967b6fd8-p9qt5" event={"ID":"fb475dbf-76e6-490c-af84-0a7e50fa3d21","Type":"ContainerDied","Data":"1d559f75d2e2b619fadd38d15d2d683d42b3b19a02065f40b7de920cebe44922"} Jul 14 10:02:10.453487 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.453406 2569 scope.go:117] "RemoveContainer" containerID="8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c" Jul 14 10:02:10.461878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.461862 2569 scope.go:117] "RemoveContainer" containerID="8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c" Jul 14 10:02:10.462163 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:02:10.462122 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c\": container with ID starting with 8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c not found: ID does not exist" containerID="8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c" Jul 14 10:02:10.462213 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.462180 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c"} err="failed to get container status \"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c\": rpc error: code = NotFound desc = could not find container \"8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c\": container with ID starting with 8eed92d5de5ed4c6e8b651d168d221598d4f5b84180d4cb6baca865992b1770c not found: ID does not exist" Jul 14 10:02:10.474218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.474196 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:02:10.477782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:10.477757 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5967b6fd8-p9qt5"] Jul 14 10:02:11.869059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:11.869017 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" path="/var/lib/kubelet/pods/fb475dbf-76e6-490c-af84-0a7e50fa3d21/volumes" Jul 14 10:02:22.913855 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:22.913825 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:02:22.917720 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:22.917696 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-55bcfc6b77-56pxm" Jul 14 10:02:24.695658 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:24.695618 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:02:24.697916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:24.697896 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcdbec16-4a3f-4206-a79f-cf02af0a3b17-metrics-certs\") pod \"network-metrics-daemon-dgbms\" (UID: \"fcdbec16-4a3f-4206-a79f-cf02af0a3b17\") " pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:02:24.786983 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:24.786956 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vm2xk\"" Jul 14 10:02:24.795146 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:24.795130 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dgbms" Jul 14 10:02:24.936036 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:24.936007 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dgbms"] Jul 14 10:02:24.939268 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:02:24.939227 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdbec16_4a3f_4206_a79f_cf02af0a3b17.slice/crio-2490ef8bb811979d22f81953fbb00a27275e5527fa41495c7dac65017782b571 WatchSource:0}: Error finding container 2490ef8bb811979d22f81953fbb00a27275e5527fa41495c7dac65017782b571: Status 404 returned error can't find the container with id 2490ef8bb811979d22f81953fbb00a27275e5527fa41495c7dac65017782b571 Jul 14 10:02:25.509794 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:25.509759 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dgbms" event={"ID":"fcdbec16-4a3f-4206-a79f-cf02af0a3b17","Type":"ContainerStarted","Data":"2490ef8bb811979d22f81953fbb00a27275e5527fa41495c7dac65017782b571"} Jul 14 10:02:26.514710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:26.514671 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dgbms" event={"ID":"fcdbec16-4a3f-4206-a79f-cf02af0a3b17","Type":"ContainerStarted","Data":"595793bc18ba4d2682bedd95f17c429d294a2f27d0bcf7071216a647643bc6c5"} Jul 14 10:02:26.515091 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:26.514729 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dgbms" event={"ID":"fcdbec16-4a3f-4206-a79f-cf02af0a3b17","Type":"ContainerStarted","Data":"72d2d5110bee7065764c2b1ecbcdc5dba93f48dc751648c9ae473c2efc83b02e"} Jul 14 10:02:26.531583 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:02:26.531518 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dgbms" podStartSLOduration=130.664919967 podStartE2EDuration="2m11.531496813s" totalImagesPullingTime="866.576846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:02:26.530285725 +0000 UTC m=+131.164349566" watchObservedRunningTime="2026-07-14 10:02:26.531496813 +0000 UTC m=+131.165560659" Jul 14 10:03:00.026962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.026927 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027218 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" containerName="registry" Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027230 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" containerName="registry" Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027262 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" containerName="console" Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027269 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" containerName="console" Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027328 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb4c5da-0352-404b-9005-b9a2ea7fdf6d" containerName="registry" Jul 14 10:03:00.027392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.027339 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb475dbf-76e6-490c-af84-0a7e50fa3d21" containerName="console" Jul 14 10:03:00.030725 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.030706 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.049872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.049676 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l8mk9\"" Jul 14 10:03:00.049872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.049676 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 10:03:00.049872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.049759 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 10:03:00.049872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.049798 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 10:03:00.049872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.049683 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 10:03:00.050490 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.050469 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 10:03:00.050631 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.050480 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 10:03:00.050714 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.050556 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 10:03:00.050994 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.050977 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 10:03:00.053210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.053187 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 10:03:00.057622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.057601 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 10:03:00.166767 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166733 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.166920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.166920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166816 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.166920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166840 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gxt4\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-kube-api-access-4gxt4\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.166920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166861 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.166920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166892 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166930 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.166954 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.167041 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.167074 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.167090 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167272 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.167115 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.167272 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.167173 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.267996 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.267944 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.267996 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268028 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268049 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268097 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268192 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.268683 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268642 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269041 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.269012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269221 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268783 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269323 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.269244 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gxt4\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-kube-api-access-4gxt4\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269323 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.269285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269440 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.269342 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.269440 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.269380 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271246 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.268753 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09340412-bcd5-4780-abb9-2188cd291d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271246 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.270972 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09340412-bcd5-4780-abb9-2188cd291d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271246 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.271171 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271448 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.271294 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271448 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.271393 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.271928 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.271952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.271945 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.272191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.272172 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.272286 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.272266 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.273133 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.273116 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09340412-bcd5-4780-abb9-2188cd291d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.277669 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.277626 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gxt4\" (UniqueName: \"kubernetes.io/projected/09340412-bcd5-4780-abb9-2188cd291d3e-kube-api-access-4gxt4\") pod \"alertmanager-main-0\" (UID: \"09340412-bcd5-4780-abb9-2188cd291d3e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.339187 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.339141 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 10:03:00.474476 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.474437 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 10:03:00.477419 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:03:00.477389 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09340412_bcd5_4780_abb9_2188cd291d3e.slice/crio-2705f638c39eaacacd886a1d809de818c88c08743c1599ce22f42ccc7f551cad WatchSource:0}: Error finding container 2705f638c39eaacacd886a1d809de818c88c08743c1599ce22f42ccc7f551cad: Status 404 returned error can't find the container with id 2705f638c39eaacacd886a1d809de818c88c08743c1599ce22f42ccc7f551cad Jul 14 10:03:00.635416 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:00.635326 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"2705f638c39eaacacd886a1d809de818c88c08743c1599ce22f42ccc7f551cad"} Jul 14 10:03:01.641126 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:01.641093 2569 generic.go:356] "Generic (PLEG): container finished" podID="09340412-bcd5-4780-abb9-2188cd291d3e" containerID="09d4fb1fe96c849724999d502f38052bfb0d0bfa57fd6170c513b482d8b664a1" exitCode=0 Jul 14 10:03:01.641484 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:01.641184 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerDied","Data":"09d4fb1fe96c849724999d502f38052bfb0d0bfa57fd6170c513b482d8b664a1"} Jul 14 10:03:02.493803 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.493766 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f846766f-6848q"] Jul 14 10:03:02.496720 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.496696 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.500659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.500345 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 10:03:02.500659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.500566 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 10:03:02.500659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.500571 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 10:03:02.501140 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.500802 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 10:03:02.501140 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.500808 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5khf2\"" Jul 14 10:03:02.501140 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.501022 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 10:03:02.507334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.507311 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 10:03:02.513173 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.513135 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f846766f-6848q"] Jul 14 10:03:02.587579 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587543 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-serving-certs-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587579 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587617 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-metrics-client-ca\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587647 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-federate-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587792 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587841 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587869 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlm5h\" (UniqueName: \"kubernetes.io/projected/2fdec8f8-2e75-4b25-88bd-b4762c959af7-kube-api-access-hlm5h\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.587962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.587905 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689311 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689278 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-federate-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689443 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689481 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689505 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlm5h\" (UniqueName: \"kubernetes.io/projected/2fdec8f8-2e75-4b25-88bd-b4762c959af7-kube-api-access-hlm5h\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689789 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689914 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689858 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-serving-certs-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.689914 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689885 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.690005 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.689922 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-metrics-client-ca\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.690698 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.690674 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-metrics-client-ca\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.690812 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.690674 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-serving-certs-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.690862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.690825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.692422 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.692401 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.692536 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.692436 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-federate-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.692536 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.692459 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-secret-telemeter-client\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.692799 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.692780 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2fdec8f8-2e75-4b25-88bd-b4762c959af7-telemeter-client-tls\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.698968 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.698941 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlm5h\" (UniqueName: \"kubernetes.io/projected/2fdec8f8-2e75-4b25-88bd-b4762c959af7-kube-api-access-hlm5h\") pod \"telemeter-client-7f846766f-6848q\" (UID: \"2fdec8f8-2e75-4b25-88bd-b4762c959af7\") " pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:02.817931 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:02.817838 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" Jul 14 10:03:03.263171 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.263132 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f846766f-6848q"] Jul 14 10:03:03.268876 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:03:03.268845 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdec8f8_2e75_4b25_88bd_b4762c959af7.slice/crio-6eb8050951d373f2e7ca4ad52cdcb6f2c0678d28187b5a7260500c6f483e630f WatchSource:0}: Error finding container 6eb8050951d373f2e7ca4ad52cdcb6f2c0678d28187b5a7260500c6f483e630f: Status 404 returned error can't find the container with id 6eb8050951d373f2e7ca4ad52cdcb6f2c0678d28187b5a7260500c6f483e630f Jul 14 10:03:03.653102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"845c98ed9a319025f957b025985b3a9ffbaf7cea2b040298c024eb8af56c8b77"} Jul 14 10:03:03.653102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653093 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"cc618db19c28fb641025658dfd7231b4c20ec578207077440bfa0a5e8a4b017c"} Jul 14 10:03:03.653350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653133 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"2e7e15815e86829c34b92c65a89581b923e8c8827173c69212268494ed9d4490"} Jul 14 10:03:03.653350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"b7cf40bb6a4332f92df4a79704f49f5f6e6bd9b0196851092aa69a8462cdefb2"} Jul 14 10:03:03.653350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653233 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"db0e68d4b10ee9000df2959eac6b67ea1eece41946c0f06c765f849a00916466"} Jul 14 10:03:03.653350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.653268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09340412-bcd5-4780-abb9-2188cd291d3e","Type":"ContainerStarted","Data":"93e917b4a7890570c9558de62b67159ec2377931a54bda0297bc585fe72d8493"} Jul 14 10:03:03.654042 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.654014 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" event={"ID":"2fdec8f8-2e75-4b25-88bd-b4762c959af7","Type":"ContainerStarted","Data":"6eb8050951d373f2e7ca4ad52cdcb6f2c0678d28187b5a7260500c6f483e630f"} Jul 14 10:03:03.680495 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:03.680440 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.11406932 podStartE2EDuration="4.680425045s" totalImagesPullingTime="2.550901609s" totalInitContainerRuntime="15.454116ms" isStatefulPod=false podCreationTimestamp="2026-07-14 10:02:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:03:03.679001342 +0000 UTC m=+168.313065183" watchObservedRunningTime="2026-07-14 10:03:03.680425045 +0000 UTC m=+168.314488885" Jul 14 10:03:05.671461 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:05.671412 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" event={"ID":"2fdec8f8-2e75-4b25-88bd-b4762c959af7","Type":"ContainerStarted","Data":"b83890b9948d5508701cad16fc8b923d75f63493090e9dd50896dfb35628cee8"} Jul 14 10:03:05.671875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:05.671511 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" event={"ID":"2fdec8f8-2e75-4b25-88bd-b4762c959af7","Type":"ContainerStarted","Data":"56296899964e1560dc3ad4b3318d72501c109bcd1037e2eb3ecdcb4f3085f59d"} Jul 14 10:03:05.671875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:05.671539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" event={"ID":"2fdec8f8-2e75-4b25-88bd-b4762c959af7","Type":"ContainerStarted","Data":"0c31ad705d6abba973e83fd386cc8ad77ce1f9656d50e3b9175e03205454ac59"} Jul 14 10:03:05.694564 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:05.694510 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f846766f-6848q" podStartSLOduration=1.7968684339999998 podStartE2EDuration="3.694495493s" totalImagesPullingTime="1.897627059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:03:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:03:05.691900436 +0000 UTC m=+170.325964276" watchObservedRunningTime="2026-07-14 10:03:05.694495493 +0000 UTC m=+170.328559331" Jul 14 10:03:06.459826 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.459788 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:03:06.462059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.462044 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.474378 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.474353 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:03:06.529883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.529846 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwglp\" (UniqueName: \"kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.529883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.529891 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.530097 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.529978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.530097 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.530014 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.530097 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.530033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.530097 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.530062 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.530247 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.530119 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.631221 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.631186 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.631485 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.631462 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwglp\" (UniqueName: \"kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.631970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.631948 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632185 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.632145 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.631970 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632405 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.632390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.632482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632593 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.632579 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.632987 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.632958 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.633640 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.633617 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.633758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.633739 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.635003 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.634973 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.635521 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.635502 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.638861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.638842 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwglp\" (UniqueName: \"kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp\") pod \"console-6bf6587469-2vrm4\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.771618 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.771532 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:06.920581 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:06.920544 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:03:06.923298 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:03:06.923271 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8bd050a_d2b6_48e3_8c1a_ad54e7698312.slice/crio-70e3e8f355afd9fb39ebb8203e328732944c70d4ac81aa24ce2c9e6382e0fd2d WatchSource:0}: Error finding container 70e3e8f355afd9fb39ebb8203e328732944c70d4ac81aa24ce2c9e6382e0fd2d: Status 404 returned error can't find the container with id 70e3e8f355afd9fb39ebb8203e328732944c70d4ac81aa24ce2c9e6382e0fd2d Jul 14 10:03:07.681934 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:07.681895 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf6587469-2vrm4" event={"ID":"a8bd050a-d2b6-48e3-8c1a-ad54e7698312","Type":"ContainerStarted","Data":"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4"} Jul 14 10:03:07.682117 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:07.681952 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf6587469-2vrm4" event={"ID":"a8bd050a-d2b6-48e3-8c1a-ad54e7698312","Type":"ContainerStarted","Data":"70e3e8f355afd9fb39ebb8203e328732944c70d4ac81aa24ce2c9e6382e0fd2d"} Jul 14 10:03:07.701195 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:07.701125 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bf6587469-2vrm4" podStartSLOduration=1.701110957 podStartE2EDuration="1.701110957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:03:07.700128471 +0000 UTC m=+172.334192325" watchObservedRunningTime="2026-07-14 10:03:07.701110957 +0000 UTC m=+172.335174797" Jul 14 10:03:16.771801 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:16.771759 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:16.772260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:16.771819 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:16.776620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:16.776599 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:17.724757 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:17.724727 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:03:17.772099 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:17.772065 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:03:42.793100 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:42.793035 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7978795d7-tvbtc" podUID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" containerName="console" containerID="cri-o://cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf" gracePeriod=15 Jul 14 10:03:43.033324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.033301 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:03:43.051018 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.050938 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.051018 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.050998 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.051574 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.051044 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.051574 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.051073 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.051574 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.051107 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpzwk\" (UniqueName: \"kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.051574 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.051463 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.052481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.052106 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:03:43.052481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.052204 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config" (OuterVolumeSpecName: "console-config") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:03:43.052650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.052492 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:03:43.052828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.052793 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:03:43.054012 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.053986 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:03:43.054924 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.054902 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk" (OuterVolumeSpecName: "kube-api-access-tpzwk") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "kube-api-access-tpzwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:03:43.152751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.152719 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert\") pod \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\" (UID: \"5e412b5c-e8cd-47e6-b73a-3662e8a3b190\") " Jul 14 10:03:43.153015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.152998 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-oauth-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.153056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.153025 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-service-ca\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.153056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.153039 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-oauth-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.153121 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.153053 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.153121 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.153067 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpzwk\" (UniqueName: \"kubernetes.io/projected/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-kube-api-access-tpzwk\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.153121 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.153080 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-trusted-ca-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.154766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.154746 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e412b5c-e8cd-47e6-b73a-3662e8a3b190" (UID: "5e412b5c-e8cd-47e6-b73a-3662e8a3b190"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:03:43.254237 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.254210 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e412b5c-e8cd-47e6-b73a-3662e8a3b190-console-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:03:43.824634 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.824596 2569 generic.go:356] "Generic (PLEG): container finished" podID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" containerID="cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf" exitCode=0 Jul 14 10:03:43.825024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.824639 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7978795d7-tvbtc" Jul 14 10:03:43.825024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.824652 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978795d7-tvbtc" event={"ID":"5e412b5c-e8cd-47e6-b73a-3662e8a3b190","Type":"ContainerDied","Data":"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf"} Jul 14 10:03:43.825024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.824705 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7978795d7-tvbtc" event={"ID":"5e412b5c-e8cd-47e6-b73a-3662e8a3b190","Type":"ContainerDied","Data":"d06b7949e016db397821d2a2e9361c92b0131d5b8b4ac81d0407ba80c8232932"} Jul 14 10:03:43.825024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.824735 2569 scope.go:117] "RemoveContainer" containerID="cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf" Jul 14 10:03:43.833172 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.833135 2569 scope.go:117] "RemoveContainer" containerID="cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf" Jul 14 10:03:43.833451 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:03:43.833434 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf\": container with ID starting with cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf not found: ID does not exist" containerID="cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf" Jul 14 10:03:43.833504 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.833459 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf"} err="failed to get container status \"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf\": rpc error: code = NotFound desc = could not find container \"cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf\": container with ID starting with cfa67b8dbeebf8046bd7f34ab82b480a3dd223f9158b24ad655eaf20119450bf not found: ID does not exist" Jul 14 10:03:43.846657 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.846634 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:03:43.855396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.855370 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7978795d7-tvbtc"] Jul 14 10:03:43.870328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:03:43.870300 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" path="/var/lib/kubelet/pods/5e412b5c-e8cd-47e6-b73a-3662e8a3b190/volumes" Jul 14 10:04:23.228563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.228529 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:04:23.228949 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.228839 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" containerName="console" Jul 14 10:04:23.228949 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.228852 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" containerName="console" Jul 14 10:04:23.228949 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.228914 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e412b5c-e8cd-47e6-b73a-3662e8a3b190" containerName="console" Jul 14 10:04:23.232289 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.232273 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.242385 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.242361 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:04:23.389237 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389237 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389239 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389257 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389394 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389411 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7p7x\" (UniqueName: \"kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.389479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.389432 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.489986 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.489881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.489986 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.489933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7p7x\" (UniqueName: \"kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.489986 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.489961 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490037 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490062 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490081 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490106 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490762 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490739 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490845 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490741 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490850 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.490992 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.490970 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.492524 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.492506 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.492580 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.492562 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.499080 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.499060 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7p7x\" (UniqueName: \"kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x\") pod \"console-664f9c678b-q2ffp\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.540965 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.540930 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:23.671142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.671059 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:04:23.673885 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:04:23.673855 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01100e5e_13d1_4059_a8b1_ad2e1c60a991.slice/crio-f8975869e8ee13be1aca73bf4b55069c8a150b3a9b2e050277cd987c0c7c7e31 WatchSource:0}: Error finding container f8975869e8ee13be1aca73bf4b55069c8a150b3a9b2e050277cd987c0c7c7e31: Status 404 returned error can't find the container with id f8975869e8ee13be1aca73bf4b55069c8a150b3a9b2e050277cd987c0c7c7e31 Jul 14 10:04:23.978772 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.978735 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f9c678b-q2ffp" event={"ID":"01100e5e-13d1-4059-a8b1-ad2e1c60a991","Type":"ContainerStarted","Data":"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405"} Jul 14 10:04:23.978952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:23.978787 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f9c678b-q2ffp" event={"ID":"01100e5e-13d1-4059-a8b1-ad2e1c60a991","Type":"ContainerStarted","Data":"f8975869e8ee13be1aca73bf4b55069c8a150b3a9b2e050277cd987c0c7c7e31"} Jul 14 10:04:24.000450 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:24.000405 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664f9c678b-q2ffp" podStartSLOduration=1.000390066 podStartE2EDuration="1.000390066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:04:23.999194913 +0000 UTC m=+248.633258755" watchObservedRunningTime="2026-07-14 10:04:24.000390066 +0000 UTC m=+248.634453969" Jul 14 10:04:33.541682 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:33.541640 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:33.542145 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:33.541930 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:33.546349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:33.546328 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:34.021974 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:34.021939 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:04:34.068363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:34.068322 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:04:59.091043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.090982 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bf6587469-2vrm4" podUID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" containerName="console" containerID="cri-o://9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4" gracePeriod=15 Jul 14 10:04:59.339429 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.339403 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:04:59.396779 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396697 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396779 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396736 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396778 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396814 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396857 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396897 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwglp\" (UniqueName: \"kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.396989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.396929 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca\") pod \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\" (UID: \"a8bd050a-d2b6-48e3-8c1a-ad54e7698312\") " Jul 14 10:04:59.397257 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397208 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:59.397309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397264 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config" (OuterVolumeSpecName: "console-config") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:59.397309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397274 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:59.397474 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397455 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-trusted-ca-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.397519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397485 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.397656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397631 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-oauth-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.397748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.397706 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:04:59.399001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.398969 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:04:59.399107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.399029 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:04:59.399382 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.399361 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp" (OuterVolumeSpecName: "kube-api-access-bwglp") pod "a8bd050a-d2b6-48e3-8c1a-ad54e7698312" (UID: "a8bd050a-d2b6-48e3-8c1a-ad54e7698312"). InnerVolumeSpecName "kube-api-access-bwglp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:04:59.498737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.498681 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-oauth-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.498737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.498728 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-console-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.498737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.498738 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwglp\" (UniqueName: \"kubernetes.io/projected/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-kube-api-access-bwglp\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:04:59.498737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:04:59.498748 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8bd050a-d2b6-48e3-8c1a-ad54e7698312-service-ca\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:05:00.122383 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.122345 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" containerID="9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4" exitCode=0 Jul 14 10:05:00.122847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.122412 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bf6587469-2vrm4" Jul 14 10:05:00.122847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.122428 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf6587469-2vrm4" event={"ID":"a8bd050a-d2b6-48e3-8c1a-ad54e7698312","Type":"ContainerDied","Data":"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4"} Jul 14 10:05:00.122847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.122481 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bf6587469-2vrm4" event={"ID":"a8bd050a-d2b6-48e3-8c1a-ad54e7698312","Type":"ContainerDied","Data":"70e3e8f355afd9fb39ebb8203e328732944c70d4ac81aa24ce2c9e6382e0fd2d"} Jul 14 10:05:00.122847 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.122501 2569 scope.go:117] "RemoveContainer" containerID="9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4" Jul 14 10:05:00.130443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.130416 2569 scope.go:117] "RemoveContainer" containerID="9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4" Jul 14 10:05:00.130692 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:05:00.130674 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4\": container with ID starting with 9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4 not found: ID does not exist" containerID="9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4" Jul 14 10:05:00.130738 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.130700 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4"} err="failed to get container status \"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4\": rpc error: code = NotFound desc = could not find container \"9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4\": container with ID starting with 9409bce7d9dbde6273ea634f929c0cc9cd1af56c3a5e8820573f7456a6b5a2e4 not found: ID does not exist" Jul 14 10:05:00.154105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.154038 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:05:00.172408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:00.172377 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bf6587469-2vrm4"] Jul 14 10:05:01.870215 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:01.870185 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" path="/var/lib/kubelet/pods/a8bd050a-d2b6-48e3-8c1a-ad54e7698312/volumes" Jul 14 10:05:15.753894 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:15.753861 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:05:15.756796 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:15.756771 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 10:05:15.757886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:15.757867 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:05:24.242189 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.242144 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt"] Jul 14 10:05:24.243554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.242491 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" containerName="console" Jul 14 10:05:24.243554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.242502 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" containerName="console" Jul 14 10:05:24.243554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.242557 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8bd050a-d2b6-48e3-8c1a-ad54e7698312" containerName="console" Jul 14 10:05:24.244675 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.244660 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.247917 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.247894 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qcz7p\"" Jul 14 10:05:24.248054 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.248000 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 10:05:24.248893 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.248879 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 10:05:24.271856 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.271822 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt"] Jul 14 10:05:24.305142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.305109 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.305342 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.305177 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.305342 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.305245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlfsf\" (UniqueName: \"kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.405715 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.405671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.405715 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.405714 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlfsf\" (UniqueName: \"kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.405963 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.405781 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.406066 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.406044 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.406135 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.406075 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.420024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.419999 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlfsf\" (UniqueName: \"kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.554198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.554097 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:24.696462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.696426 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt"] Jul 14 10:05:24.699622 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:05:24.699594 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda500eb2_1cf8_49ff_a817_d9e4833da6f3.slice/crio-b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87 WatchSource:0}: Error finding container b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87: Status 404 returned error can't find the container with id b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87 Jul 14 10:05:24.702016 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:24.701997 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 10:05:25.221061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:25.221026 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" event={"ID":"da500eb2-1cf8-49ff-a817-d9e4833da6f3","Type":"ContainerStarted","Data":"b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87"} Jul 14 10:05:32.253390 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:32.253344 2569 generic.go:356] "Generic (PLEG): container finished" podID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerID="d2981bd45321cc4529033953fb9e5b4c03e41094a3963339c1a1daf24d35f3e7" exitCode=0 Jul 14 10:05:32.253854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:32.253430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" event={"ID":"da500eb2-1cf8-49ff-a817-d9e4833da6f3","Type":"ContainerDied","Data":"d2981bd45321cc4529033953fb9e5b4c03e41094a3963339c1a1daf24d35f3e7"} Jul 14 10:05:35.267392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:35.267350 2569 generic.go:356] "Generic (PLEG): container finished" podID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerID="9194e68697878e3f98491217cd064b20a390ddfec4c57118736abe2215cb5a88" exitCode=0 Jul 14 10:05:35.267793 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:35.267447 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" event={"ID":"da500eb2-1cf8-49ff-a817-d9e4833da6f3","Type":"ContainerDied","Data":"9194e68697878e3f98491217cd064b20a390ddfec4c57118736abe2215cb5a88"} Jul 14 10:05:42.301031 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:42.300939 2569 generic.go:356] "Generic (PLEG): container finished" podID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerID="1543ca86a0f00d49853fbd70509aa111e0ac24cf32f089f668844f838cfbe517" exitCode=0 Jul 14 10:05:42.301031 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:42.300979 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" event={"ID":"da500eb2-1cf8-49ff-a817-d9e4833da6f3","Type":"ContainerDied","Data":"1543ca86a0f00d49853fbd70509aa111e0ac24cf32f089f668844f838cfbe517"} Jul 14 10:05:43.429970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.429939 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:43.471754 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.471718 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util\") pod \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " Jul 14 10:05:43.471905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.471769 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle\") pod \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " Jul 14 10:05:43.471905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.471830 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlfsf\" (UniqueName: \"kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf\") pod \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\" (UID: \"da500eb2-1cf8-49ff-a817-d9e4833da6f3\") " Jul 14 10:05:43.472471 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.472447 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle" (OuterVolumeSpecName: "bundle") pod "da500eb2-1cf8-49ff-a817-d9e4833da6f3" (UID: "da500eb2-1cf8-49ff-a817-d9e4833da6f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.474023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.473995 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf" (OuterVolumeSpecName: "kube-api-access-xlfsf") pod "da500eb2-1cf8-49ff-a817-d9e4833da6f3" (UID: "da500eb2-1cf8-49ff-a817-d9e4833da6f3"). InnerVolumeSpecName "kube-api-access-xlfsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:05:43.475768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.475748 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util" (OuterVolumeSpecName: "util") pod "da500eb2-1cf8-49ff-a817-d9e4833da6f3" (UID: "da500eb2-1cf8-49ff-a817-d9e4833da6f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:05:43.573265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.573170 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.573265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.573210 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da500eb2-1cf8-49ff-a817-d9e4833da6f3-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:05:43.573265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:43.573224 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlfsf\" (UniqueName: \"kubernetes.io/projected/da500eb2-1cf8-49ff-a817-d9e4833da6f3-kube-api-access-xlfsf\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:05:44.310994 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:44.310964 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" Jul 14 10:05:44.310994 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:44.310973 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qs4wt" event={"ID":"da500eb2-1cf8-49ff-a817-d9e4833da6f3","Type":"ContainerDied","Data":"b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87"} Jul 14 10:05:44.311229 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:44.311023 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9e27453983f623c69b149ef7051dcc9a68a495f9f0935111bdf7bfe78ccea87" Jul 14 10:05:52.574665 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574628 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl"] Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574929 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="extract" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574939 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="extract" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574958 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="pull" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574963 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="pull" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574974 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="util" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.574979 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="util" Jul 14 10:05:52.575056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.575027 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="da500eb2-1cf8-49ff-a817-d9e4833da6f3" containerName="extract" Jul 14 10:05:52.578279 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.578262 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.581659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.581639 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 10:05:52.581947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.581926 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9slrk\"" Jul 14 10:05:52.581947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.581942 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 10:05:52.597047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.597021 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl"] Jul 14 10:05:52.649440 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.649407 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1696d4e4-a328-4718-afe8-e4e034a72d86-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.649601 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.649482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgb9\" (UniqueName: \"kubernetes.io/projected/1696d4e4-a328-4718-afe8-e4e034a72d86-kube-api-access-smgb9\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.749917 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.749864 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1696d4e4-a328-4718-afe8-e4e034a72d86-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.750095 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.749969 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgb9\" (UniqueName: \"kubernetes.io/projected/1696d4e4-a328-4718-afe8-e4e034a72d86-kube-api-access-smgb9\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.750398 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.750377 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1696d4e4-a328-4718-afe8-e4e034a72d86-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.802201 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.802139 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgb9\" (UniqueName: \"kubernetes.io/projected/1696d4e4-a328-4718-afe8-e4e034a72d86-kube-api-access-smgb9\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x9dl\" (UID: \"1696d4e4-a328-4718-afe8-e4e034a72d86\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:52.887335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:52.887250 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" Jul 14 10:05:53.033223 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:53.033197 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl"] Jul 14 10:05:53.034087 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:05:53.034057 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1696d4e4_a328_4718_afe8_e4e034a72d86.slice/crio-8d8eb96291869b2465ca82316ed9a4c881df667800b9022b5798f1a0166bd232 WatchSource:0}: Error finding container 8d8eb96291869b2465ca82316ed9a4c881df667800b9022b5798f1a0166bd232: Status 404 returned error can't find the container with id 8d8eb96291869b2465ca82316ed9a4c881df667800b9022b5798f1a0166bd232 Jul 14 10:05:53.349924 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:53.349887 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" event={"ID":"1696d4e4-a328-4718-afe8-e4e034a72d86","Type":"ContainerStarted","Data":"8d8eb96291869b2465ca82316ed9a4c881df667800b9022b5798f1a0166bd232"} Jul 14 10:05:55.361092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:55.361055 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" event={"ID":"1696d4e4-a328-4718-afe8-e4e034a72d86","Type":"ContainerStarted","Data":"0d898b2db76b0577597d867df1878a2c7bce468780c8e4329bdcaa4ccfd3a36a"} Jul 14 10:05:55.397129 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:55.397075 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x9dl" podStartSLOduration=1.805187495 podStartE2EDuration="3.397058969s" totalImagesPullingTime="1.591871474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:05:55.39580543 +0000 UTC m=+340.029869273" watchObservedRunningTime="2026-07-14 10:05:55.397058969 +0000 UTC m=+340.031122814" Jul 14 10:05:56.373971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.373939 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx"] Jul 14 10:05:56.376846 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.376823 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.383700 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.383677 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qcz7p\"" Jul 14 10:05:56.385706 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.385686 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 10:05:56.388193 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.388177 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 10:05:56.404315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.404293 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx"] Jul 14 10:05:56.479362 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.479327 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.479546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.479391 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwfv\" (UniqueName: \"kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.479546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.479488 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.580285 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.580239 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwfv\" (UniqueName: \"kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.580480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.580323 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.580480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.580362 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.580808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.580777 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.580808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.580792 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.596631 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.596605 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwfv\" (UniqueName: \"kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.685903 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.685867 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:05:56.740894 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.740866 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tjlg2"] Jul 14 10:05:56.744060 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.744037 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.753112 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.752977 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cbgbs\"" Jul 14 10:05:56.753112 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.752987 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 10:05:56.753112 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.753005 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 10:05:56.765807 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.765748 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tjlg2"] Jul 14 10:05:56.826972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.826948 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx"] Jul 14 10:05:56.829442 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:05:56.829416 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2955ea34_2a65_4eb2_826e_457da48775dc.slice/crio-efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682 WatchSource:0}: Error finding container efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682: Status 404 returned error can't find the container with id efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682 Jul 14 10:05:56.882509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.882482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.882607 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.882576 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8flq\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-kube-api-access-d8flq\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.983479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.983385 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.983479 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.983467 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8flq\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-kube-api-access-d8flq\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.994048 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.994025 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:56.999854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:56.999837 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8flq\" (UniqueName: \"kubernetes.io/projected/387276a6-d5e8-44ec-8963-52371c25ad2d-kube-api-access-d8flq\") pod \"cert-manager-webhook-6486ffd997-tjlg2\" (UID: \"387276a6-d5e8-44ec-8963-52371c25ad2d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:57.064509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.064462 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:05:57.207049 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.207002 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tjlg2"] Jul 14 10:05:57.211926 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:05:57.211900 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod387276a6_d5e8_44ec_8963_52371c25ad2d.slice/crio-bbc728e118446bd1ba4fb4eef02f676a17a1858ffa245582648661beca5baabf WatchSource:0}: Error finding container bbc728e118446bd1ba4fb4eef02f676a17a1858ffa245582648661beca5baabf: Status 404 returned error can't find the container with id bbc728e118446bd1ba4fb4eef02f676a17a1858ffa245582648661beca5baabf Jul 14 10:05:57.370987 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.370900 2569 generic.go:356] "Generic (PLEG): container finished" podID="2955ea34-2a65-4eb2-826e-457da48775dc" containerID="9e4233024d8f64ca0584ee5a6d29730c60efe9ceb9148bfa99b10eae7d1d6ba5" exitCode=0 Jul 14 10:05:57.371145 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.370984 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerDied","Data":"9e4233024d8f64ca0584ee5a6d29730c60efe9ceb9148bfa99b10eae7d1d6ba5"} Jul 14 10:05:57.371145 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.371066 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerStarted","Data":"efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682"} Jul 14 10:05:57.372319 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:57.372291 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" event={"ID":"387276a6-d5e8-44ec-8963-52371c25ad2d","Type":"ContainerStarted","Data":"bbc728e118446bd1ba4fb4eef02f676a17a1858ffa245582648661beca5baabf"} Jul 14 10:05:59.154651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.154618 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h"] Jul 14 10:05:59.158101 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.158078 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.161345 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.161322 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jxbq7\"" Jul 14 10:05:59.176142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.176118 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h"] Jul 14 10:05:59.304537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.304499 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.304720 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.304551 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bznwh\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-kube-api-access-bznwh\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.406027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.405950 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.406027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.405993 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bznwh\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-kube-api-access-bznwh\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.438334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.438303 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.439993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.439965 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bznwh\" (UniqueName: \"kubernetes.io/projected/880018a5-66b2-4d81-b5a2-ad219ac943e3-kube-api-access-bznwh\") pod \"cert-manager-cainjector-559cdd8c87-xrr7h\" (UID: \"880018a5-66b2-4d81-b5a2-ad219ac943e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:05:59.468821 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:05:59.468776 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" Jul 14 10:06:00.392650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:00.392571 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" event={"ID":"387276a6-d5e8-44ec-8963-52371c25ad2d","Type":"ContainerStarted","Data":"6b8890b51f578e0733e2e7d2cfa44544ad4828613362919469b6cd34ef071ea5"} Jul 14 10:06:00.392650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:00.392629 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:06:00.396087 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:00.396061 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerStarted","Data":"b8a3056e57b044b9387c786058ecd26f50900cf7ea65f1dfd3577a05650ed27c"} Jul 14 10:06:00.430111 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:00.430063 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" podStartSLOduration=1.346529504 podStartE2EDuration="4.430046386s" totalImagesPullingTime="3.083516882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:00.428219743 +0000 UTC m=+345.062283584" watchObservedRunningTime="2026-07-14 10:06:00.430046386 +0000 UTC m=+345.064110227" Jul 14 10:06:00.461353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:00.461233 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h"] Jul 14 10:06:00.516028 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:00.515997 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod880018a5_66b2_4d81_b5a2_ad219ac943e3.slice/crio-adb6dfb9ede7a49e3d746c4c60835eeae77e4828acc79a2a41f7048d7038bb97 WatchSource:0}: Error finding container adb6dfb9ede7a49e3d746c4c60835eeae77e4828acc79a2a41f7048d7038bb97: Status 404 returned error can't find the container with id adb6dfb9ede7a49e3d746c4c60835eeae77e4828acc79a2a41f7048d7038bb97 Jul 14 10:06:01.401709 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:01.401679 2569 generic.go:356] "Generic (PLEG): container finished" podID="2955ea34-2a65-4eb2-826e-457da48775dc" containerID="b8a3056e57b044b9387c786058ecd26f50900cf7ea65f1dfd3577a05650ed27c" exitCode=0 Jul 14 10:06:01.402165 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:01.401753 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerDied","Data":"b8a3056e57b044b9387c786058ecd26f50900cf7ea65f1dfd3577a05650ed27c"} Jul 14 10:06:01.403379 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:01.403323 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" event={"ID":"880018a5-66b2-4d81-b5a2-ad219ac943e3","Type":"ContainerStarted","Data":"719fdea4ec44f35aeb70a04fffde5a51eaca1c5ca7f0f6d052e8425fd04b72bf"} Jul 14 10:06:01.403379 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:01.403358 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" event={"ID":"880018a5-66b2-4d81-b5a2-ad219ac943e3","Type":"ContainerStarted","Data":"adb6dfb9ede7a49e3d746c4c60835eeae77e4828acc79a2a41f7048d7038bb97"} Jul 14 10:06:01.460768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:01.460709 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xrr7h" podStartSLOduration=2.460687826 podStartE2EDuration="2.460687826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:01.457965212 +0000 UTC m=+346.092029055" watchObservedRunningTime="2026-07-14 10:06:01.460687826 +0000 UTC m=+346.094751668" Jul 14 10:06:02.410076 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:02.410039 2569 generic.go:356] "Generic (PLEG): container finished" podID="2955ea34-2a65-4eb2-826e-457da48775dc" containerID="646e46d184b765a69c85a989c06ea4280247344c1a842a71be9dd08635a9bfda" exitCode=0 Jul 14 10:06:02.410546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:02.410126 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerDied","Data":"646e46d184b765a69c85a989c06ea4280247344c1a842a71be9dd08635a9bfda"} Jul 14 10:06:03.538895 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.538869 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:06:03.644437 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.644403 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvwfv\" (UniqueName: \"kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv\") pod \"2955ea34-2a65-4eb2-826e-457da48775dc\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " Jul 14 10:06:03.644601 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.644511 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util\") pod \"2955ea34-2a65-4eb2-826e-457da48775dc\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " Jul 14 10:06:03.644601 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.644536 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle\") pod \"2955ea34-2a65-4eb2-826e-457da48775dc\" (UID: \"2955ea34-2a65-4eb2-826e-457da48775dc\") " Jul 14 10:06:03.644983 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.644951 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle" (OuterVolumeSpecName: "bundle") pod "2955ea34-2a65-4eb2-826e-457da48775dc" (UID: "2955ea34-2a65-4eb2-826e-457da48775dc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:06:03.646688 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.646664 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv" (OuterVolumeSpecName: "kube-api-access-rvwfv") pod "2955ea34-2a65-4eb2-826e-457da48775dc" (UID: "2955ea34-2a65-4eb2-826e-457da48775dc"). InnerVolumeSpecName "kube-api-access-rvwfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:06:03.648867 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.648841 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util" (OuterVolumeSpecName: "util") pod "2955ea34-2a65-4eb2-826e-457da48775dc" (UID: "2955ea34-2a65-4eb2-826e-457da48775dc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:06:03.745367 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.745334 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:03.745367 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.745362 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2955ea34-2a65-4eb2-826e-457da48775dc-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:03.745367 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:03.745371 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvwfv\" (UniqueName: \"kubernetes.io/projected/2955ea34-2a65-4eb2-826e-457da48775dc-kube-api-access-rvwfv\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:04.421234 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:04.421202 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" Jul 14 10:06:04.421234 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:04.421213 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flncnx" event={"ID":"2955ea34-2a65-4eb2-826e-457da48775dc","Type":"ContainerDied","Data":"efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682"} Jul 14 10:06:04.421443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:04.421261 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="efca082690b84fdc31bf755598301a192e05e972e7314c5924106443b5e6a682" Jul 14 10:06:06.405351 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:06.405317 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tjlg2" Jul 14 10:06:07.809667 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.809571 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rpsms"] Jul 14 10:06:07.810116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810056 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="pull" Jul 14 10:06:07.810116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810075 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="pull" Jul 14 10:06:07.810256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810129 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="extract" Jul 14 10:06:07.810256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810139 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="extract" Jul 14 10:06:07.810256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810170 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="util" Jul 14 10:06:07.810256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810179 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="util" Jul 14 10:06:07.810440 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.810266 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2955ea34-2a65-4eb2-826e-457da48775dc" containerName="extract" Jul 14 10:06:07.812170 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.812133 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:07.815774 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.815749 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-b54h6\"" Jul 14 10:06:07.829120 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.829095 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rpsms"] Jul 14 10:06:07.984791 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.984757 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-bound-sa-token\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:07.984972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:07.984834 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msx2f\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-kube-api-access-msx2f\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.085782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.085693 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msx2f\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-kube-api-access-msx2f\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.085782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.085746 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-bound-sa-token\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.105535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.105505 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-bound-sa-token\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.105676 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.105511 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msx2f\" (UniqueName: \"kubernetes.io/projected/2a590bbe-68e5-4549-a701-ee65cc98f540-kube-api-access-msx2f\") pod \"cert-manager-55b4d954d5-rpsms\" (UID: \"2a590bbe-68e5-4549-a701-ee65cc98f540\") " pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.123323 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.123296 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rpsms" Jul 14 10:06:08.272700 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.272669 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rpsms"] Jul 14 10:06:08.276263 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:08.276234 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a590bbe_68e5_4549_a701_ee65cc98f540.slice/crio-367a4f7d070964e0286043d60fccfd7c197b11acdfc79c782169172efa6b2f5d WatchSource:0}: Error finding container 367a4f7d070964e0286043d60fccfd7c197b11acdfc79c782169172efa6b2f5d: Status 404 returned error can't find the container with id 367a4f7d070964e0286043d60fccfd7c197b11acdfc79c782169172efa6b2f5d Jul 14 10:06:08.441412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.441373 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rpsms" event={"ID":"2a590bbe-68e5-4549-a701-ee65cc98f540","Type":"ContainerStarted","Data":"d4426acf6aff635897cf1a372e38ffa375d8f6d74e0d7efec32cf8ee320e6047"} Jul 14 10:06:08.441618 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.441425 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rpsms" event={"ID":"2a590bbe-68e5-4549-a701-ee65cc98f540","Type":"ContainerStarted","Data":"367a4f7d070964e0286043d60fccfd7c197b11acdfc79c782169172efa6b2f5d"} Jul 14 10:06:08.465236 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:08.465177 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rpsms" podStartSLOduration=1.465131747 podStartE2EDuration="1.465131747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:08.463884691 +0000 UTC m=+353.097948531" watchObservedRunningTime="2026-07-14 10:06:08.465131747 +0000 UTC m=+353.099195612" Jul 14 10:06:10.162378 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.162337 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st"] Jul 14 10:06:10.165135 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.165116 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.172293 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.172274 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 10:06:10.172388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.172274 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 10:06:10.175207 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.175192 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6v279\"" Jul 14 10:06:10.182987 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.182960 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st"] Jul 14 10:06:10.303457 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.303423 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbgck\" (UniqueName: \"kubernetes.io/projected/ae11fcad-1c6a-4879-95bb-becff826d505-kube-api-access-gbgck\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.303631 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.303486 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae11fcad-1c6a-4879-95bb-becff826d505-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.404254 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.404205 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbgck\" (UniqueName: \"kubernetes.io/projected/ae11fcad-1c6a-4879-95bb-becff826d505-kube-api-access-gbgck\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.404404 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.404371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae11fcad-1c6a-4879-95bb-becff826d505-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.404749 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.404732 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae11fcad-1c6a-4879-95bb-becff826d505-tmp\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.418474 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.418425 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbgck\" (UniqueName: \"kubernetes.io/projected/ae11fcad-1c6a-4879-95bb-becff826d505-kube-api-access-gbgck\") pod \"openshift-lws-operator-fd8ccff4c-j99st\" (UID: \"ae11fcad-1c6a-4879-95bb-becff826d505\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.474105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.474083 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" Jul 14 10:06:10.618962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:10.618924 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st"] Jul 14 10:06:10.621775 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:10.621751 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae11fcad_1c6a_4879_95bb_becff826d505.slice/crio-bfd861ff3d967e898631f27738eb2b6986aeee9bf3830488c5abc3b7c9591c0e WatchSource:0}: Error finding container bfd861ff3d967e898631f27738eb2b6986aeee9bf3830488c5abc3b7c9591c0e: Status 404 returned error can't find the container with id bfd861ff3d967e898631f27738eb2b6986aeee9bf3830488c5abc3b7c9591c0e Jul 14 10:06:11.457903 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:11.457852 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" event={"ID":"ae11fcad-1c6a-4879-95bb-becff826d505","Type":"ContainerStarted","Data":"bfd861ff3d967e898631f27738eb2b6986aeee9bf3830488c5abc3b7c9591c0e"} Jul 14 10:06:13.468952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:13.468910 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" event={"ID":"ae11fcad-1c6a-4879-95bb-becff826d505","Type":"ContainerStarted","Data":"76c3a56691ff299c4eef5084ca0c8212a6cb0ab115dfd5658dde50813de93c84"} Jul 14 10:06:13.492103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:13.492056 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-j99st" podStartSLOduration=1.250471659 podStartE2EDuration="3.492040854s" totalImagesPullingTime="2.241569195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:13.489915633 +0000 UTC m=+358.123979473" watchObservedRunningTime="2026-07-14 10:06:13.492040854 +0000 UTC m=+358.126104695" Jul 14 10:06:16.451467 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.451433 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w"] Jul 14 10:06:16.454457 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.454440 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.461905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.461884 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qcz7p\"" Jul 14 10:06:16.461905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.461902 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 10:06:16.462926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.462910 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 10:06:16.474947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.474927 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w"] Jul 14 10:06:16.556468 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.556437 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.556645 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.556482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rztg\" (UniqueName: \"kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.556645 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.556528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.657837 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.657795 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.657837 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.657844 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rztg\" (UniqueName: \"kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.658090 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.657885 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.658238 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.658219 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.658296 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.658238 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.669536 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.669506 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rztg\" (UniqueName: \"kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.763928 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.763839 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:16.914275 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:16.914255 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w"] Jul 14 10:06:16.916792 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:16.916766 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7667e09_ad58_4804_858e_ea897d5fc3ce.slice/crio-1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301 WatchSource:0}: Error finding container 1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301: Status 404 returned error can't find the container with id 1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301 Jul 14 10:06:17.489033 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:17.489000 2569 generic.go:356] "Generic (PLEG): container finished" podID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerID="78cd1c19edfdc17797481b2a017c3877767edf1785ccfd650a5885ff6f2eda03" exitCode=0 Jul 14 10:06:17.489512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:17.489081 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" event={"ID":"f7667e09-ad58-4804-858e-ea897d5fc3ce","Type":"ContainerDied","Data":"78cd1c19edfdc17797481b2a017c3877767edf1785ccfd650a5885ff6f2eda03"} Jul 14 10:06:17.489512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:17.489130 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" event={"ID":"f7667e09-ad58-4804-858e-ea897d5fc3ce","Type":"ContainerStarted","Data":"1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301"} Jul 14 10:06:18.495498 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:18.495469 2569 generic.go:356] "Generic (PLEG): container finished" podID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerID="ea5d8e67d23885aea48667ec47729b8e563a01adc5bbb265730564331d6513a3" exitCode=0 Jul 14 10:06:18.495861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:18.495559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" event={"ID":"f7667e09-ad58-4804-858e-ea897d5fc3ce","Type":"ContainerDied","Data":"ea5d8e67d23885aea48667ec47729b8e563a01adc5bbb265730564331d6513a3"} Jul 14 10:06:19.502802 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:19.502768 2569 generic.go:356] "Generic (PLEG): container finished" podID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerID="61b53582bf745ce336efd96c270b7da9716e057d6dec11c0ba082a95f5a9939d" exitCode=0 Jul 14 10:06:19.503202 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:19.502861 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" event={"ID":"f7667e09-ad58-4804-858e-ea897d5fc3ce","Type":"ContainerDied","Data":"61b53582bf745ce336efd96c270b7da9716e057d6dec11c0ba082a95f5a9939d"} Jul 14 10:06:20.641599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.641574 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:20.691038 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.691008 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle\") pod \"f7667e09-ad58-4804-858e-ea897d5fc3ce\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " Jul 14 10:06:20.691204 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.691057 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rztg\" (UniqueName: \"kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg\") pod \"f7667e09-ad58-4804-858e-ea897d5fc3ce\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " Jul 14 10:06:20.691204 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.691116 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util\") pod \"f7667e09-ad58-4804-858e-ea897d5fc3ce\" (UID: \"f7667e09-ad58-4804-858e-ea897d5fc3ce\") " Jul 14 10:06:20.691781 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.691757 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle" (OuterVolumeSpecName: "bundle") pod "f7667e09-ad58-4804-858e-ea897d5fc3ce" (UID: "f7667e09-ad58-4804-858e-ea897d5fc3ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:06:20.693261 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.693234 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg" (OuterVolumeSpecName: "kube-api-access-5rztg") pod "f7667e09-ad58-4804-858e-ea897d5fc3ce" (UID: "f7667e09-ad58-4804-858e-ea897d5fc3ce"). InnerVolumeSpecName "kube-api-access-5rztg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:06:20.696872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.696850 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util" (OuterVolumeSpecName: "util") pod "f7667e09-ad58-4804-858e-ea897d5fc3ce" (UID: "f7667e09-ad58-4804-858e-ea897d5fc3ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:06:20.792059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.791952 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:20.792059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.792004 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7667e09-ad58-4804-858e-ea897d5fc3ce-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:20.792059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:20.792014 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rztg\" (UniqueName: \"kubernetes.io/projected/f7667e09-ad58-4804-858e-ea897d5fc3ce-kube-api-access-5rztg\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:06:21.517119 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:21.517096 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" Jul 14 10:06:21.517310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:21.517095 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlz2w" event={"ID":"f7667e09-ad58-4804-858e-ea897d5fc3ce","Type":"ContainerDied","Data":"1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301"} Jul 14 10:06:21.517310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:21.517193 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b9f137226aae06d06d3078222c45de59269e513f0e9a1f2e88f97c748b40301" Jul 14 10:06:29.176202 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176168 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj"] Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176488 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="extract" Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176498 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="extract" Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176513 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="pull" Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176518 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="pull" Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176525 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="util" Jul 14 10:06:29.176561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176530 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="util" Jul 14 10:06:29.176775 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.176594 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7667e09-ad58-4804-858e-ea897d5fc3ce" containerName="extract" Jul 14 10:06:29.182895 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.182870 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.186322 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.186303 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 10:06:29.186322 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.186318 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7mwsq\"" Jul 14 10:06:29.186889 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.186870 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 10:06:29.187036 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.187020 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 10:06:29.187097 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.187025 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 10:06:29.210799 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.210772 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj"] Jul 14 10:06:29.268810 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.268775 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.268810 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.268811 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.269018 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.268860 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-manifests\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.269018 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.268875 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-tmp\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.269018 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.268904 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhtrz\" (UniqueName: \"kubernetes.io/projected/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-kube-api-access-rhtrz\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.369791 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.369758 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-manifests\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.369791 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.369791 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-tmp\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.370016 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.369827 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhtrz\" (UniqueName: \"kubernetes.io/projected/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-kube-api-access-rhtrz\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.370016 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.369878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.370016 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.369902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.370285 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.370263 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-manifests\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.372199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.372177 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-tmp\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.372338 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.372321 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.372443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.372427 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.378861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.378843 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhtrz\" (UniqueName: \"kubernetes.io/projected/e69b7b2c-d086-489a-b005-fa5ba1fdfff6-kube-api-access-rhtrz\") pod \"opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj\" (UID: \"e69b7b2c-d086-489a-b005-fa5ba1fdfff6\") " pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.493500 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.493463 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:29.640512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:29.640483 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj"] Jul 14 10:06:29.641755 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:29.641729 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69b7b2c_d086_489a_b005_fa5ba1fdfff6.slice/crio-fe7f82a0aff64269cc7af7127fe49f9547dadcabc0d263eaa46b77afc7f5c2ac WatchSource:0}: Error finding container fe7f82a0aff64269cc7af7127fe49f9547dadcabc0d263eaa46b77afc7f5c2ac: Status 404 returned error can't find the container with id fe7f82a0aff64269cc7af7127fe49f9547dadcabc0d263eaa46b77afc7f5c2ac Jul 14 10:06:30.561110 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:30.561024 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" event={"ID":"e69b7b2c-d086-489a-b005-fa5ba1fdfff6","Type":"ContainerStarted","Data":"fe7f82a0aff64269cc7af7127fe49f9547dadcabc0d263eaa46b77afc7f5c2ac"} Jul 14 10:06:33.577015 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:33.576981 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" event={"ID":"e69b7b2c-d086-489a-b005-fa5ba1fdfff6","Type":"ContainerStarted","Data":"0b987f3b1b83ff596ae51ae03e44ea96aef57efe351bd9227b6992a4f31b88db"} Jul 14 10:06:34.583491 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:34.583450 2569 generic.go:356] "Generic (PLEG): container finished" podID="e69b7b2c-d086-489a-b005-fa5ba1fdfff6" containerID="0b987f3b1b83ff596ae51ae03e44ea96aef57efe351bd9227b6992a4f31b88db" exitCode=0 Jul 14 10:06:34.583862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:34.583497 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" event={"ID":"e69b7b2c-d086-489a-b005-fa5ba1fdfff6","Type":"ContainerDied","Data":"0b987f3b1b83ff596ae51ae03e44ea96aef57efe351bd9227b6992a4f31b88db"} Jul 14 10:06:35.590703 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:35.590670 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" event={"ID":"e69b7b2c-d086-489a-b005-fa5ba1fdfff6","Type":"ContainerStarted","Data":"f544d83f12e92df41c49b5352682568bddb7e38ddfb5e21766b0662a29cea1d8"} Jul 14 10:06:35.591086 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:35.590756 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:35.618376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:35.618328 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" podStartSLOduration=1.93510887 podStartE2EDuration="6.618313131s" totalImagesPullingTime="3.582137993s" totalInitContainerRuntime="1.101066268s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:35.6180583 +0000 UTC m=+380.252122135" watchObservedRunningTime="2026-07-14 10:06:35.618313131 +0000 UTC m=+380.252376971" Jul 14 10:06:46.597487 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:46.597457 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj" Jul 14 10:06:49.495470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.495435 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-589999c949-twspv"] Jul 14 10:06:49.503734 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.503715 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.508295 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.508275 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 10:06:49.508416 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.508282 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 10:06:49.508692 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.508641 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hbwws\"" Jul 14 10:06:49.511945 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.509123 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 10:06:49.526554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.526528 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-589999c949-twspv"] Jul 14 10:06:49.655457 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.655426 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntdcv\" (UniqueName: \"kubernetes.io/projected/8e1f51cb-c70a-40bf-8d6c-793ccc477345-kube-api-access-ntdcv\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.655622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.655465 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.655622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.655508 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-metrics-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.655622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.655533 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1f51cb-c70a-40bf-8d6c-793ccc477345-manager-config\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.756776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.756685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntdcv\" (UniqueName: \"kubernetes.io/projected/8e1f51cb-c70a-40bf-8d6c-793ccc477345-kube-api-access-ntdcv\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.756776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.756735 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.756776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.756761 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-metrics-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.756981 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.756785 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1f51cb-c70a-40bf-8d6c-793ccc477345-manager-config\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.757470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.757448 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1f51cb-c70a-40bf-8d6c-793ccc477345-manager-config\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.759275 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.759249 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-metrics-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.759358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.759283 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1f51cb-c70a-40bf-8d6c-793ccc477345-cert\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.766092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.766061 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntdcv\" (UniqueName: \"kubernetes.io/projected/8e1f51cb-c70a-40bf-8d6c-793ccc477345-kube-api-access-ntdcv\") pod \"lws-controller-manager-589999c949-twspv\" (UID: \"8e1f51cb-c70a-40bf-8d6c-793ccc477345\") " pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.817981 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.817938 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:49.960960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:49.960895 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-589999c949-twspv"] Jul 14 10:06:49.963793 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:49.963761 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1f51cb_c70a_40bf_8d6c_793ccc477345.slice/crio-366d1bfdc9a4b20c26ad01e34de03bc7483ee3f047082f895c95cab11cd528c1 WatchSource:0}: Error finding container 366d1bfdc9a4b20c26ad01e34de03bc7483ee3f047082f895c95cab11cd528c1: Status 404 returned error can't find the container with id 366d1bfdc9a4b20c26ad01e34de03bc7483ee3f047082f895c95cab11cd528c1 Jul 14 10:06:50.663127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:50.663084 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" event={"ID":"8e1f51cb-c70a-40bf-8d6c-793ccc477345","Type":"ContainerStarted","Data":"366d1bfdc9a4b20c26ad01e34de03bc7483ee3f047082f895c95cab11cd528c1"} Jul 14 10:06:51.668653 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:51.668621 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" event={"ID":"8e1f51cb-c70a-40bf-8d6c-793ccc477345","Type":"ContainerStarted","Data":"e6359125ed98c788476bebf946a453b21ce896ad8192e3578f64454551950778"} Jul 14 10:06:51.669053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:51.668728 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:06:51.686934 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:51.686882 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" podStartSLOduration=1.110636269 podStartE2EDuration="2.686861288s" totalImagesPullingTime="1.576225019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:06:51.686484633 +0000 UTC m=+396.320548475" watchObservedRunningTime="2026-07-14 10:06:51.686861288 +0000 UTC m=+396.320925131" Jul 14 10:06:56.822109 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.822072 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4"] Jul 14 10:06:56.825993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.825972 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.828844 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.828823 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-v8wcm\"" Jul 14 10:06:56.828966 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.828824 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 10:06:56.829199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.829182 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 10:06:56.829433 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.829419 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 10:06:56.838417 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.838395 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4"] Jul 14 10:06:56.921674 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921639 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b302a7-97d9-44b4-b065-cf6c43bf330f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.921674 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnzx5\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-kube-api-access-mnzx5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.921896 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921770 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.921896 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.921896 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921857 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.922006 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921894 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.922006 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921957 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.922006 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.921980 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:56.922133 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:56.922005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022492 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022454 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022492 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b302a7-97d9-44b4-b065-cf6c43bf330f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022553 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnzx5\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-kube-api-access-mnzx5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022589 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022631 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.022701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022701 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022887 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023001 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.022909 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.023324 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b302a7-97d9-44b4-b065-cf6c43bf330f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023483 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.023405 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.023545 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.023479 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.025087 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.025064 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.025361 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.025336 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.032521 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.032496 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.032619 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.032539 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnzx5\" (UniqueName: \"kubernetes.io/projected/29b302a7-97d9-44b4-b065-cf6c43bf330f-kube-api-access-mnzx5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gzbt4\" (UID: \"29b302a7-97d9-44b4-b065-cf6c43bf330f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.137333 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.137246 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:06:57.332350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.332269 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4"] Jul 14 10:06:57.336065 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:57.336036 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b302a7_97d9_44b4_b065_cf6c43bf330f.slice/crio-24c58be25fb3095a41c9e7e3ee13cf2478492083ca93d0be054c965bda381076 WatchSource:0}: Error finding container 24c58be25fb3095a41c9e7e3ee13cf2478492083ca93d0be054c965bda381076: Status 404 returned error can't find the container with id 24c58be25fb3095a41c9e7e3ee13cf2478492083ca93d0be054c965bda381076 Jul 14 10:06:57.386100 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.386063 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb"] Jul 14 10:06:57.390738 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.390679 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.394017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.393985 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-psm7t\"" Jul 14 10:06:57.394464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.394445 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 10:06:57.394561 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.394446 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 10:06:57.394622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.394552 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 10:06:57.409420 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.409384 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb"] Jul 14 10:06:57.527862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.527824 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.527862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.527861 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g6nn\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-kube-api-access-4g6nn\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.528061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.527897 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.528061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.527975 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.528061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.528010 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.528061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.528053 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.528212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.528081 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6051ad4d-18c2-4c64-80e4-998423db2c9b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.628725 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.628687 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.628725 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.628724 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.628972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.628749 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.628972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.628890 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.628972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.628954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6051ad4d-18c2-4c64-80e4-998423db2c9b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.629118 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.629047 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.629118 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.629076 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g6nn\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-kube-api-access-4g6nn\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.629839 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.629816 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.631968 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.631930 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.632085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.631988 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.632085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.632003 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6051ad4d-18c2-4c64-80e4-998423db2c9b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.632085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.632019 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6051ad4d-18c2-4c64-80e4-998423db2c9b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.641753 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.641688 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.643228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.643205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g6nn\" (UniqueName: \"kubernetes.io/projected/6051ad4d-18c2-4c64-80e4-998423db2c9b-kube-api-access-4g6nn\") pod \"istiod-openshift-gateway-94bb8fbfd-45qvb\" (UID: \"6051ad4d-18c2-4c64-80e4-998423db2c9b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.698119 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.698086 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" event={"ID":"29b302a7-97d9-44b4-b065-cf6c43bf330f","Type":"ContainerStarted","Data":"24c58be25fb3095a41c9e7e3ee13cf2478492083ca93d0be054c965bda381076"} Jul 14 10:06:57.701297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.701276 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:06:57.873647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:57.873530 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb"] Jul 14 10:06:57.877768 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:06:57.877727 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6051ad4d_18c2_4c64_80e4_998423db2c9b.slice/crio-fcc9aabb92d838840dd9dfc5be72efdd135d5108ed3ec2b8246f61aa357a4c6e WatchSource:0}: Error finding container fcc9aabb92d838840dd9dfc5be72efdd135d5108ed3ec2b8246f61aa357a4c6e: Status 404 returned error can't find the container with id fcc9aabb92d838840dd9dfc5be72efdd135d5108ed3ec2b8246f61aa357a4c6e Jul 14 10:06:58.705736 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:58.705696 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" event={"ID":"6051ad4d-18c2-4c64-80e4-998423db2c9b","Type":"ContainerStarted","Data":"fcc9aabb92d838840dd9dfc5be72efdd135d5108ed3ec2b8246f61aa357a4c6e"} Jul 14 10:06:59.758265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:59.758223 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:06:59.758654 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:59.758323 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:06:59.758654 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:06:59.758379 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:07:00.729860 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:00.729826 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" event={"ID":"29b302a7-97d9-44b4-b065-cf6c43bf330f","Type":"ContainerStarted","Data":"746ad34966af08bddd9c0efc3e7480bfaeb4458db239a0b44c7ed38e3bcc66c5"} Jul 14 10:07:00.770818 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:00.770757 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" podStartSLOduration=2.351459721 podStartE2EDuration="4.770741074s" totalImagesPullingTime="2.419281353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:07:00.767360658 +0000 UTC m=+405.401424520" watchObservedRunningTime="2026-07-14 10:07:00.770741074 +0000 UTC m=+405.404804915" Jul 14 10:07:01.138260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.138174 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:07:01.143941 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.143904 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:07:01.577459 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.577423 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:07:01.577590 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.577499 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:07:01.737228 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.737192 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" event={"ID":"6051ad4d-18c2-4c64-80e4-998423db2c9b","Type":"ContainerStarted","Data":"b6871f17fefba524f28b04e766d3e045a3f958b6dfad44a1d549549b7c921ed4"} Jul 14 10:07:01.737431 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.737404 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:07:01.737512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.737457 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:07:01.738357 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.738331 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gzbt4" Jul 14 10:07:01.757458 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:01.757271 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" podStartSLOduration=1.061462058 podStartE2EDuration="4.757253892s" totalImagesPullingTime="3.695791834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:06:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:07:01.757034428 +0000 UTC m=+406.391098285" watchObservedRunningTime="2026-07-14 10:07:01.757253892 +0000 UTC m=+406.391317731" Jul 14 10:07:02.675525 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:02.675497 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-589999c949-twspv" Jul 14 10:07:02.744105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:07:02.744077 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-45qvb" Jul 14 10:08:05.849964 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.849926 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2"] Jul 14 10:08:05.852680 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.852663 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:05.855651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.855630 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 10:08:05.855750 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.855664 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 10:08:05.855750 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.855669 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sxmr7\"" Jul 14 10:08:05.862509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.862488 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2"] Jul 14 10:08:05.942894 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.942863 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:05.943064 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.942924 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4s7m\" (UniqueName: \"kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:05.943064 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:05.943014 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.043798 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.043751 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4s7m\" (UniqueName: \"kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.043975 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.043843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.043975 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.043881 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.044305 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.044285 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.044360 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.044332 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.053608 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.053575 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4s7m\" (UniqueName: \"kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.162737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.162647 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:06.246439 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.246406 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm"] Jul 14 10:08:06.249174 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.249127 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.259316 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.259285 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm"] Jul 14 10:08:06.307856 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.307832 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2"] Jul 14 10:08:06.309776 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:08:06.309741 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea330f78_f9fb_4c94_8d13_9398fa111f68.slice/crio-555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b WatchSource:0}: Error finding container 555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b: Status 404 returned error can't find the container with id 555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b Jul 14 10:08:06.347199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.347168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s945\" (UniqueName: \"kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.347334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.347210 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.347334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.347245 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.448687 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.448651 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s945\" (UniqueName: \"kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.448687 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.448689 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.448872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.448721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.449088 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.449072 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.449125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.449070 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.457981 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.457956 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s945\" (UniqueName: \"kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.564010 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.563966 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:06.646252 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.646225 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k"] Jul 14 10:08:06.649831 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.649809 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.660217 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.660189 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k"] Jul 14 10:08:06.696726 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.696619 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm"] Jul 14 10:08:06.699336 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:08:06.699311 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf87a83_6b48_49fd_9356_437ef3d17848.slice/crio-c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8 WatchSource:0}: Error finding container c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8: Status 404 returned error can't find the container with id c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8 Jul 14 10:08:06.751608 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.751571 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979zv\" (UniqueName: \"kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.751716 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.751690 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.751778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.751736 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.853023 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.852989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.853387 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.853032 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.853387 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.853071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979zv\" (UniqueName: \"kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.853387 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.853360 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.853505 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.853394 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.862645 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.862618 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-979zv\" (UniqueName: \"kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:06.962294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:06.962221 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:07.056126 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.056094 2569 generic.go:356] "Generic (PLEG): container finished" podID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerID="0c81b585365a2a25c39d14f9628a610cc49aa84c86206254cd93dcbf8be9237a" exitCode=0 Jul 14 10:08:07.056312 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.056195 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" event={"ID":"cdf87a83-6b48-49fd-9356-437ef3d17848","Type":"ContainerDied","Data":"0c81b585365a2a25c39d14f9628a610cc49aa84c86206254cd93dcbf8be9237a"} Jul 14 10:08:07.056312 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.056266 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" event={"ID":"cdf87a83-6b48-49fd-9356-437ef3d17848","Type":"ContainerStarted","Data":"c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8"} Jul 14 10:08:07.058054 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.058004 2569 generic.go:356] "Generic (PLEG): container finished" podID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerID="57adb471a55d9fe78506a2ea335d39bf0ad70b058e0e46b2535313e19567c856" exitCode=0 Jul 14 10:08:07.058193 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.058107 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" event={"ID":"ea330f78-f9fb-4c94-8d13-9398fa111f68","Type":"ContainerDied","Data":"57adb471a55d9fe78506a2ea335d39bf0ad70b058e0e46b2535313e19567c856"} Jul 14 10:08:07.058193 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.058144 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" event={"ID":"ea330f78-f9fb-4c94-8d13-9398fa111f68","Type":"ContainerStarted","Data":"555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b"} Jul 14 10:08:07.065520 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.063537 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr"] Jul 14 10:08:07.073253 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.070420 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.081497 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.081468 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr"] Jul 14 10:08:07.155785 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.155748 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dvks\" (UniqueName: \"kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.155950 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.155808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.155950 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.155910 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.257068 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.256980 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dvks\" (UniqueName: \"kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.257068 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.257027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.257336 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.257074 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.257459 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.257441 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.257540 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.257520 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.265886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.265862 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dvks\" (UniqueName: \"kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.306644 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.306612 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k"] Jul 14 10:08:07.309239 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:08:07.309210 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode217d45f_032e_461e_b28c_fe195281508b.slice/crio-2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179 WatchSource:0}: Error finding container 2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179: Status 404 returned error can't find the container with id 2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179 Jul 14 10:08:07.388684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.388658 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:07.517402 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:07.517376 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr"] Jul 14 10:08:07.519230 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:08:07.519207 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70e6f00a_2c33_40b6_8c80_1f1ba76b2ba8.slice/crio-af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c WatchSource:0}: Error finding container af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c: Status 404 returned error can't find the container with id af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c Jul 14 10:08:08.065537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.065502 2569 generic.go:356] "Generic (PLEG): container finished" podID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerID="36ffb4657876b532b052cadcebb3901320372fec6625369798d48012bb8d1487" exitCode=0 Jul 14 10:08:08.065945 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.065584 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" event={"ID":"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8","Type":"ContainerDied","Data":"36ffb4657876b532b052cadcebb3901320372fec6625369798d48012bb8d1487"} Jul 14 10:08:08.065945 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.065632 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" event={"ID":"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8","Type":"ContainerStarted","Data":"af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c"} Jul 14 10:08:08.067307 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.067282 2569 generic.go:356] "Generic (PLEG): container finished" podID="e217d45f-032e-461e-b28c-fe195281508b" containerID="f88625907ea5fb7407140a6d4a890925e68ecefbab5325f3fa8810f4b683c482" exitCode=0 Jul 14 10:08:08.067458 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.067347 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" event={"ID":"e217d45f-032e-461e-b28c-fe195281508b","Type":"ContainerDied","Data":"f88625907ea5fb7407140a6d4a890925e68ecefbab5325f3fa8810f4b683c482"} Jul 14 10:08:08.067458 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.067389 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" event={"ID":"e217d45f-032e-461e-b28c-fe195281508b","Type":"ContainerStarted","Data":"2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179"} Jul 14 10:08:08.069462 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.069441 2569 generic.go:356] "Generic (PLEG): container finished" podID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerID="495ec3b6706c9e4a7920fe7e91f3245f5cf5a2462f65c4cb14d46eaa7518561f" exitCode=0 Jul 14 10:08:08.069565 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.069542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" event={"ID":"cdf87a83-6b48-49fd-9356-437ef3d17848","Type":"ContainerDied","Data":"495ec3b6706c9e4a7920fe7e91f3245f5cf5a2462f65c4cb14d46eaa7518561f"} Jul 14 10:08:08.071547 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.071529 2569 generic.go:356] "Generic (PLEG): container finished" podID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerID="4b4c4a479cfbf4987d41417831540c28189bb3f406285527fe329f33e62cc724" exitCode=0 Jul 14 10:08:08.071622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:08.071607 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" event={"ID":"ea330f78-f9fb-4c94-8d13-9398fa111f68","Type":"ContainerDied","Data":"4b4c4a479cfbf4987d41417831540c28189bb3f406285527fe329f33e62cc724"} Jul 14 10:08:09.078644 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.078609 2569 generic.go:356] "Generic (PLEG): container finished" podID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerID="aeb6f09701e89140aeaa69cdfc61f810e00f82b17d077b3bf963b26e59150ca1" exitCode=0 Jul 14 10:08:09.078980 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.078681 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" event={"ID":"ea330f78-f9fb-4c94-8d13-9398fa111f68","Type":"ContainerDied","Data":"aeb6f09701e89140aeaa69cdfc61f810e00f82b17d077b3bf963b26e59150ca1"} Jul 14 10:08:09.080531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.080508 2569 generic.go:356] "Generic (PLEG): container finished" podID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerID="4bb593bd18e07ea9124386ccadc321fadfc9b13c542c8d2a42824f2af052f6a7" exitCode=0 Jul 14 10:08:09.080613 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.080587 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" event={"ID":"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8","Type":"ContainerDied","Data":"4bb593bd18e07ea9124386ccadc321fadfc9b13c542c8d2a42824f2af052f6a7"} Jul 14 10:08:09.082468 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.082446 2569 generic.go:356] "Generic (PLEG): container finished" podID="e217d45f-032e-461e-b28c-fe195281508b" containerID="bb700348d29798eaece9d995a0a3ef6bd0c00118a22a04aa1c27410680f3f677" exitCode=0 Jul 14 10:08:09.082572 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.082530 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" event={"ID":"e217d45f-032e-461e-b28c-fe195281508b","Type":"ContainerDied","Data":"bb700348d29798eaece9d995a0a3ef6bd0c00118a22a04aa1c27410680f3f677"} Jul 14 10:08:09.084865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.084778 2569 generic.go:356] "Generic (PLEG): container finished" podID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerID="c5b4d0a76725c5a92ac52e833707710d1880a44db693e3ce8ea73b362053cc85" exitCode=0 Jul 14 10:08:09.084865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:09.084829 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" event={"ID":"cdf87a83-6b48-49fd-9356-437ef3d17848","Type":"ContainerDied","Data":"c5b4d0a76725c5a92ac52e833707710d1880a44db693e3ce8ea73b362053cc85"} Jul 14 10:08:10.091996 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.091959 2569 generic.go:356] "Generic (PLEG): container finished" podID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerID="bd5a15ff9be5f3ba5302d155c1a94d3aee18ddc6788e60147b1c0c489f4efcb2" exitCode=0 Jul 14 10:08:10.092486 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.092057 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" event={"ID":"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8","Type":"ContainerDied","Data":"bd5a15ff9be5f3ba5302d155c1a94d3aee18ddc6788e60147b1c0c489f4efcb2"} Jul 14 10:08:10.094063 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.094040 2569 generic.go:356] "Generic (PLEG): container finished" podID="e217d45f-032e-461e-b28c-fe195281508b" containerID="258ffd7a334c9fe420168fe8ff1fabaad8149c1728e502e5192116f451f75151" exitCode=0 Jul 14 10:08:10.094212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.094113 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" event={"ID":"e217d45f-032e-461e-b28c-fe195281508b","Type":"ContainerDied","Data":"258ffd7a334c9fe420168fe8ff1fabaad8149c1728e502e5192116f451f75151"} Jul 14 10:08:10.227630 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.227606 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:10.263583 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.263555 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:10.283529 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.283501 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util\") pod \"ea330f78-f9fb-4c94-8d13-9398fa111f68\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " Jul 14 10:08:10.283691 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.283548 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle\") pod \"ea330f78-f9fb-4c94-8d13-9398fa111f68\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " Jul 14 10:08:10.283691 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.283588 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4s7m\" (UniqueName: \"kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m\") pod \"ea330f78-f9fb-4c94-8d13-9398fa111f68\" (UID: \"ea330f78-f9fb-4c94-8d13-9398fa111f68\") " Jul 14 10:08:10.284170 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.284134 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle" (OuterVolumeSpecName: "bundle") pod "ea330f78-f9fb-4c94-8d13-9398fa111f68" (UID: "ea330f78-f9fb-4c94-8d13-9398fa111f68"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:10.286061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.286036 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m" (OuterVolumeSpecName: "kube-api-access-x4s7m") pod "ea330f78-f9fb-4c94-8d13-9398fa111f68" (UID: "ea330f78-f9fb-4c94-8d13-9398fa111f68"). InnerVolumeSpecName "kube-api-access-x4s7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:08:10.289577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.289552 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util" (OuterVolumeSpecName: "util") pod "ea330f78-f9fb-4c94-8d13-9398fa111f68" (UID: "ea330f78-f9fb-4c94-8d13-9398fa111f68"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:10.384298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384208 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util\") pod \"cdf87a83-6b48-49fd-9356-437ef3d17848\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " Jul 14 10:08:10.384298 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384281 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle\") pod \"cdf87a83-6b48-49fd-9356-437ef3d17848\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " Jul 14 10:08:10.384492 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384324 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s945\" (UniqueName: \"kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945\") pod \"cdf87a83-6b48-49fd-9356-437ef3d17848\" (UID: \"cdf87a83-6b48-49fd-9356-437ef3d17848\") " Jul 14 10:08:10.384546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384506 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:10.384546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384516 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea330f78-f9fb-4c94-8d13-9398fa111f68-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:10.384546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384525 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4s7m\" (UniqueName: \"kubernetes.io/projected/ea330f78-f9fb-4c94-8d13-9398fa111f68-kube-api-access-x4s7m\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:10.384792 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.384764 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle" (OuterVolumeSpecName: "bundle") pod "cdf87a83-6b48-49fd-9356-437ef3d17848" (UID: "cdf87a83-6b48-49fd-9356-437ef3d17848"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:10.386494 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.386475 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945" (OuterVolumeSpecName: "kube-api-access-8s945") pod "cdf87a83-6b48-49fd-9356-437ef3d17848" (UID: "cdf87a83-6b48-49fd-9356-437ef3d17848"). InnerVolumeSpecName "kube-api-access-8s945". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:08:10.389724 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.389705 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util" (OuterVolumeSpecName: "util") pod "cdf87a83-6b48-49fd-9356-437ef3d17848" (UID: "cdf87a83-6b48-49fd-9356-437ef3d17848"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:10.485072 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.485034 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:10.485072 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.485067 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cdf87a83-6b48-49fd-9356-437ef3d17848-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:10.485072 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:10.485080 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s945\" (UniqueName: \"kubernetes.io/projected/cdf87a83-6b48-49fd-9356-437ef3d17848-kube-api-access-8s945\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.101994 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.101968 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" Jul 14 10:08:11.102433 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.102008 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2" event={"ID":"ea330f78-f9fb-4c94-8d13-9398fa111f68","Type":"ContainerDied","Data":"555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b"} Jul 14 10:08:11.102433 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.102060 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="555f11aac67371ca8b21b0bdb81cd766793150b5ea35ee0b2a95f1f63566b09b" Jul 14 10:08:11.103989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.103961 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" Jul 14 10:08:11.103989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.103971 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm" event={"ID":"cdf87a83-6b48-49fd-9356-437ef3d17848","Type":"ContainerDied","Data":"c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8"} Jul 14 10:08:11.104204 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.104005 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c86cb8a3286fbd19e881981e7e34c6aae8d8938091d9ec496075683d568050b8" Jul 14 10:08:11.243599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.243576 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:11.269899 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.269872 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:11.394659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394575 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle\") pod \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " Jul 14 10:08:11.394659 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394654 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-979zv\" (UniqueName: \"kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv\") pod \"e217d45f-032e-461e-b28c-fe195281508b\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " Jul 14 10:08:11.394886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394676 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle\") pod \"e217d45f-032e-461e-b28c-fe195281508b\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " Jul 14 10:08:11.394886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394707 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util\") pod \"e217d45f-032e-461e-b28c-fe195281508b\" (UID: \"e217d45f-032e-461e-b28c-fe195281508b\") " Jul 14 10:08:11.394886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394727 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dvks\" (UniqueName: \"kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks\") pod \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " Jul 14 10:08:11.394886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.394770 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util\") pod \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\" (UID: \"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8\") " Jul 14 10:08:11.395175 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.395133 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle" (OuterVolumeSpecName: "bundle") pod "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" (UID: "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:11.395401 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.395355 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle" (OuterVolumeSpecName: "bundle") pod "e217d45f-032e-461e-b28c-fe195281508b" (UID: "e217d45f-032e-461e-b28c-fe195281508b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:11.396922 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.396900 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv" (OuterVolumeSpecName: "kube-api-access-979zv") pod "e217d45f-032e-461e-b28c-fe195281508b" (UID: "e217d45f-032e-461e-b28c-fe195281508b"). InnerVolumeSpecName "kube-api-access-979zv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:08:11.397037 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.396918 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks" (OuterVolumeSpecName: "kube-api-access-4dvks") pod "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" (UID: "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8"). InnerVolumeSpecName "kube-api-access-4dvks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:08:11.400741 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.400719 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util" (OuterVolumeSpecName: "util") pod "e217d45f-032e-461e-b28c-fe195281508b" (UID: "e217d45f-032e-461e-b28c-fe195281508b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:11.400836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.400802 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util" (OuterVolumeSpecName: "util") pod "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" (UID: "70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:08:11.496099 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496065 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.496099 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496092 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dvks\" (UniqueName: \"kubernetes.io/projected/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-kube-api-access-4dvks\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.496099 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496102 2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-util\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.496342 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496111 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.496342 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496119 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-979zv\" (UniqueName: \"kubernetes.io/projected/e217d45f-032e-461e-b28c-fe195281508b-kube-api-access-979zv\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:11.496342 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:11.496127 2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217d45f-032e-461e-b28c-fe195281508b-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:12.111318 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.111274 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" event={"ID":"70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8","Type":"ContainerDied","Data":"af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c"} Jul 14 10:08:12.111318 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.111323 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af4a6e8e690b6ed690b7b46496503c85cf0a5ddaf4281c5e8904d151a871998c" Jul 14 10:08:12.111748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.111282 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr" Jul 14 10:08:12.113344 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.113327 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" Jul 14 10:08:12.113482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.113357 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k" event={"ID":"e217d45f-032e-461e-b28c-fe195281508b","Type":"ContainerDied","Data":"2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179"} Jul 14 10:08:12.113482 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:12.113402 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ebe509cc0b1acad42646c910c1fdbb7c307c8a708eb8729600aa80cf890f179" Jul 14 10:08:34.262459 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:34.262425 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:08:59.283195 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.283134 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-664f9c678b-q2ffp" podUID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" containerName="console" containerID="cri-o://2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405" gracePeriod=15 Jul 14 10:08:59.532988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.532963 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:08:59.629722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629629 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.629722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629682 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7p7x\" (UniqueName: \"kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.629722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629720 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.630011 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629777 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.630011 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629895 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.630011 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629987 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.630199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630044 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca\") pod \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\" (UID: \"01100e5e-13d1-4059-a8b1-ad2e1c60a991\") " Jul 14 10:08:59.630199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.629986 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config" (OuterVolumeSpecName: "console-config") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:08:59.630199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630186 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:08:59.630358 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630269 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:08:59.630415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630386 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-trusted-ca-bundle\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.630415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630406 2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.630481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630420 2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-oauth-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.630481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.630462 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca" (OuterVolumeSpecName: "service-ca") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:08:59.631906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.631884 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x" (OuterVolumeSpecName: "kube-api-access-j7p7x") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "kube-api-access-j7p7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:08:59.632410 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.632393 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:08:59.632485 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.632409 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "01100e5e-13d1-4059-a8b1-ad2e1c60a991" (UID: "01100e5e-13d1-4059-a8b1-ad2e1c60a991"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:08:59.731197 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.731136 2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-serving-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.731197 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.731189 2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01100e5e-13d1-4059-a8b1-ad2e1c60a991-service-ca\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.731197 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.731199 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7p7x\" (UniqueName: \"kubernetes.io/projected/01100e5e-13d1-4059-a8b1-ad2e1c60a991-kube-api-access-j7p7x\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:08:59.731437 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:08:59.731211 2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01100e5e-13d1-4059-a8b1-ad2e1c60a991-console-oauth-config\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:09:00.362686 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.362645 2569 generic.go:356] "Generic (PLEG): container finished" podID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" containerID="2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405" exitCode=0 Jul 14 10:09:00.363198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.362699 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f9c678b-q2ffp" event={"ID":"01100e5e-13d1-4059-a8b1-ad2e1c60a991","Type":"ContainerDied","Data":"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405"} Jul 14 10:09:00.363198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.362709 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664f9c678b-q2ffp" Jul 14 10:09:00.363198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.362748 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664f9c678b-q2ffp" event={"ID":"01100e5e-13d1-4059-a8b1-ad2e1c60a991","Type":"ContainerDied","Data":"f8975869e8ee13be1aca73bf4b55069c8a150b3a9b2e050277cd987c0c7c7e31"} Jul 14 10:09:00.363198 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.362774 2569 scope.go:117] "RemoveContainer" containerID="2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405" Jul 14 10:09:00.371299 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.371279 2569 scope.go:117] "RemoveContainer" containerID="2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405" Jul 14 10:09:00.371553 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:09:00.371532 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405\": container with ID starting with 2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405 not found: ID does not exist" containerID="2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405" Jul 14 10:09:00.371601 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.371561 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405"} err="failed to get container status \"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405\": rpc error: code = NotFound desc = could not find container \"2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405\": container with ID starting with 2e05d1ed555854696913490c5c4d492bc689bc615788aa99d768ee2bfa250405 not found: ID does not exist" Jul 14 10:09:00.382363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.382342 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:09:00.386123 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:00.386102 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664f9c678b-q2ffp"] Jul 14 10:09:01.870131 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:01.870093 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" path="/var/lib/kubelet/pods/01100e5e-13d1-4059-a8b1-ad2e1c60a991/volumes" Jul 14 10:09:20.162168 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162117 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c"] Jul 14 10:09:20.162522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162496 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="extract" Jul 14 10:09:20.162522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162509 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="extract" Jul 14 10:09:20.162522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162517 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="pull" Jul 14 10:09:20.162522 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162523 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="pull" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162533 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="util" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162538 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="util" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162547 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="pull" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162554 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="pull" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162571 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="extract" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162577 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="extract" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162589 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" containerName="console" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162595 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" containerName="console" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162601 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="util" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162606 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="util" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162610 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="pull" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162615 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="pull" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162621 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="extract" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162636 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="extract" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162649 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="util" Jul 14 10:09:20.162651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162654 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="util" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162661 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="pull" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162666 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="pull" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162678 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="util" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162683 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="util" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162688 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="extract" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162693 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="extract" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162753 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="01100e5e-13d1-4059-a8b1-ad2e1c60a991" containerName="console" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162763 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf87a83-6b48-49fd-9356-437ef3d17848" containerName="extract" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162769 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8" containerName="extract" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162775 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea330f78-f9fb-4c94-8d13-9398fa111f68" containerName="extract" Jul 14 10:09:20.163178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.162782 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e217d45f-032e-461e-b28c-fe195281508b" containerName="extract" Jul 14 10:09:20.165599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.165582 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.168334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.168314 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-75w2h\"" Jul 14 10:09:20.177788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.177760 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c"] Jul 14 10:09:20.313860 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.313830 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.313891 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.313914 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdce0995-348a-4885-825f-b040a0f4001c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.313940 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314053 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.313983 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.314076 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.314114 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tnv\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-kube-api-access-f2tnv\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.314142 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.314297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.314198 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdce0995-348a-4885-825f-b040a0f4001c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415195 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415245 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdce0995-348a-4885-825f-b040a0f4001c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415267 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415314 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415399 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tnv\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-kube-api-access-f2tnv\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415424 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdce0995-348a-4885-825f-b040a0f4001c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415646 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.415800 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415708 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.416186 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.415974 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.416186 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.416042 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.416268 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.416236 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdce0995-348a-4885-825f-b040a0f4001c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.417701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.417680 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdce0995-348a-4885-825f-b040a0f4001c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.417784 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.417724 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdce0995-348a-4885-825f-b040a0f4001c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.423988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.423964 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.424089 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.424029 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tnv\" (UniqueName: \"kubernetes.io/projected/fdce0995-348a-4885-825f-b040a0f4001c-kube-api-access-f2tnv\") pod \"maas-default-gateway-openshift-default-56654dcf7-btd4c\" (UID: \"fdce0995-348a-4885-825f-b040a0f4001c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.476916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.476887 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:20.610112 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.610089 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c"] Jul 14 10:09:20.611745 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:09:20.611720 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdce0995_348a_4885_825f_b040a0f4001c.slice/crio-bd2f5874cc364e8e98a97f1a1b68226d63a3b76da9c67d3df6919e576c5af0f7 WatchSource:0}: Error finding container bd2f5874cc364e8e98a97f1a1b68226d63a3b76da9c67d3df6919e576c5af0f7: Status 404 returned error can't find the container with id bd2f5874cc364e8e98a97f1a1b68226d63a3b76da9c67d3df6919e576c5af0f7 Jul 14 10:09:20.613862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.613830 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:09:20.613946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.613903 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:09:20.613946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:20.613930 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:09:21.476990 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:21.476951 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" event={"ID":"fdce0995-348a-4885-825f-b040a0f4001c","Type":"ContainerStarted","Data":"85808a59c8d1fdbd06ce6bb65759ebd2de7219db8759aa4e1ec1c0f927c542ef"} Jul 14 10:09:21.477464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:21.477005 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" event={"ID":"fdce0995-348a-4885-825f-b040a0f4001c","Type":"ContainerStarted","Data":"bd2f5874cc364e8e98a97f1a1b68226d63a3b76da9c67d3df6919e576c5af0f7"} Jul 14 10:09:21.477464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:21.477026 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:21.481920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:21.481889 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:21.501227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:21.501178 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" podStartSLOduration=1.501162757 podStartE2EDuration="1.501162757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:09:21.498885675 +0000 UTC m=+546.132949517" watchObservedRunningTime="2026-07-14 10:09:21.501162757 +0000 UTC m=+546.135226589" Jul 14 10:09:22.481991 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:22.481963 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:09:22.483077 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:09:22.483057 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:10:13.627553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.627522 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:13.630865 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.630847 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.633696 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.633668 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 10:10:13.633823 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.633700 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z8s6p\"" Jul 14 10:10:13.633823 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.633781 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 10:10:13.641541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.641512 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:13.690773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.690741 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v84b\" (UniqueName: \"kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.690948 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.690795 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.791529 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.791495 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v84b\" (UniqueName: \"kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.791702 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.791534 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.794201 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.794177 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.801242 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.801219 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v84b\" (UniqueName: \"kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b\") pod \"maas-controller-cf8fbfdc5-qqnnk\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.804444 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.804420 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:10:13.807732 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.807709 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:13.822021 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.821994 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:10:13.891990 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.891909 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdjdj\" (UniqueName: \"kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:13.891990 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.891966 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:13.942876 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.942841 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:13.993240 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.993194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdjdj\" (UniqueName: \"kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:13.993374 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.993306 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:13.996613 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:13.996589 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:14.002823 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.002798 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdjdj\" (UniqueName: \"kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj\") pod \"maas-controller-667d7fccf6-v2n2c\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:14.074774 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.074749 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:14.076525 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:10:14.076500 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c6ea95_e3f6_458d_b205_3dea01c7e479.slice/crio-b89bb2181ef98fe19952fa51a0930eb360fd15a2f3466850dba88256e1d75910 WatchSource:0}: Error finding container b89bb2181ef98fe19952fa51a0930eb360fd15a2f3466850dba88256e1d75910: Status 404 returned error can't find the container with id b89bb2181ef98fe19952fa51a0930eb360fd15a2f3466850dba88256e1d75910 Jul 14 10:10:14.117461 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.117415 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:14.248092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.248067 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:10:14.249788 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:10:14.249767 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8156d66_50bb_4c71_be35_0c90f2be1003.slice/crio-3092516bdbca5da7ddfdcf32b1e64f930cdbde9bb061e74ad6c8041835d04f3a WatchSource:0}: Error finding container 3092516bdbca5da7ddfdcf32b1e64f930cdbde9bb061e74ad6c8041835d04f3a: Status 404 returned error can't find the container with id 3092516bdbca5da7ddfdcf32b1e64f930cdbde9bb061e74ad6c8041835d04f3a Jul 14 10:10:14.755906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.755873 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" event={"ID":"f8156d66-50bb-4c71-be35-0c90f2be1003","Type":"ContainerStarted","Data":"3092516bdbca5da7ddfdcf32b1e64f930cdbde9bb061e74ad6c8041835d04f3a"} Jul 14 10:10:14.757238 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:14.757211 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" event={"ID":"35c6ea95-e3f6-458d-b205-3dea01c7e479","Type":"ContainerStarted","Data":"b89bb2181ef98fe19952fa51a0930eb360fd15a2f3466850dba88256e1d75910"} Jul 14 10:10:15.829588 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:15.829554 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:10:15.830067 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:15.829800 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:10:16.451873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.451845 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 10:10:16.774742 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.774651 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" event={"ID":"f8156d66-50bb-4c71-be35-0c90f2be1003","Type":"ContainerStarted","Data":"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097"} Jul 14 10:10:16.774902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.774750 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:16.776120 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.776094 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" event={"ID":"35c6ea95-e3f6-458d-b205-3dea01c7e479","Type":"ContainerStarted","Data":"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7"} Jul 14 10:10:16.776280 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.776216 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:16.796145 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.796098 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" podStartSLOduration=1.591462366 podStartE2EDuration="3.796086966s" totalImagesPullingTime="2.2046246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:10:16.795249116 +0000 UTC m=+601.429312968" watchObservedRunningTime="2026-07-14 10:10:16.796086966 +0000 UTC m=+601.430150841" Jul 14 10:10:16.818803 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:16.818767 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" podStartSLOduration=1.44789164 podStartE2EDuration="3.818751219s" totalImagesPullingTime="2.370859579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:10:16.816068989 +0000 UTC m=+601.450132834" watchObservedRunningTime="2026-07-14 10:10:16.818751219 +0000 UTC m=+601.452815062" Jul 14 10:10:27.787524 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:27.787488 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:10:27.789978 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:27.787563 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:27.860613 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:27.860574 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:27.860814 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:27.860780 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" podUID="35c6ea95-e3f6-458d-b205-3dea01c7e479" containerName="manager" containerID="cri-o://69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7" gracePeriod=30 Jul 14 10:10:28.115202 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.115178 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:28.221637 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.221601 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v84b\" (UniqueName: \"kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b\") pod \"35c6ea95-e3f6-458d-b205-3dea01c7e479\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " Jul 14 10:10:28.221846 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.221760 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert\") pod \"35c6ea95-e3f6-458d-b205-3dea01c7e479\" (UID: \"35c6ea95-e3f6-458d-b205-3dea01c7e479\") " Jul 14 10:10:28.223877 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.223847 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b" (OuterVolumeSpecName: "kube-api-access-9v84b") pod "35c6ea95-e3f6-458d-b205-3dea01c7e479" (UID: "35c6ea95-e3f6-458d-b205-3dea01c7e479"). InnerVolumeSpecName "kube-api-access-9v84b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:10:28.223970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.223890 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "35c6ea95-e3f6-458d-b205-3dea01c7e479" (UID: "35c6ea95-e3f6-458d-b205-3dea01c7e479"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:10:28.323049 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.323015 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35c6ea95-e3f6-458d-b205-3dea01c7e479-webhook-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:10:28.323049 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.323042 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v84b\" (UniqueName: \"kubernetes.io/projected/35c6ea95-e3f6-458d-b205-3dea01c7e479-kube-api-access-9v84b\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:10:28.839296 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.839259 2569 generic.go:356] "Generic (PLEG): container finished" podID="35c6ea95-e3f6-458d-b205-3dea01c7e479" containerID="69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7" exitCode=0 Jul 14 10:10:28.839756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.839308 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" Jul 14 10:10:28.839756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.839353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" event={"ID":"35c6ea95-e3f6-458d-b205-3dea01c7e479","Type":"ContainerDied","Data":"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7"} Jul 14 10:10:28.839756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.839419 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-cf8fbfdc5-qqnnk" event={"ID":"35c6ea95-e3f6-458d-b205-3dea01c7e479","Type":"ContainerDied","Data":"b89bb2181ef98fe19952fa51a0930eb360fd15a2f3466850dba88256e1d75910"} Jul 14 10:10:28.839756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.839434 2569 scope.go:117] "RemoveContainer" containerID="69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7" Jul 14 10:10:28.848519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.848499 2569 scope.go:117] "RemoveContainer" containerID="69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7" Jul 14 10:10:28.848770 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:10:28.848750 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7\": container with ID starting with 69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7 not found: ID does not exist" containerID="69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7" Jul 14 10:10:28.848835 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.848779 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7"} err="failed to get container status \"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7\": rpc error: code = NotFound desc = could not find container \"69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7\": container with ID starting with 69e63294ba0721ea3c5706b0621d371b7fa9ac71d2f4538d3d609f82b5993be7 not found: ID does not exist" Jul 14 10:10:28.862380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.862357 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:28.866182 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:28.866160 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-cf8fbfdc5-qqnnk"] Jul 14 10:10:29.870548 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:29.870517 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c6ea95-e3f6-458d-b205-3dea01c7e479" path="/var/lib/kubelet/pods/35c6ea95-e3f6-458d-b205-3dea01c7e479/volumes" Jul 14 10:10:57.993408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.993374 2569 generic.go:356] "Generic (PLEG): container finished" podID="fdce0995-348a-4885-825f-b040a0f4001c" containerID="85808a59c8d1fdbd06ce6bb65759ebd2de7219db8759aa4e1ec1c0f927c542ef" exitCode=0 Jul 14 10:10:57.993872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.993453 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" event={"ID":"fdce0995-348a-4885-825f-b040a0f4001c","Type":"ContainerDied","Data":"85808a59c8d1fdbd06ce6bb65759ebd2de7219db8759aa4e1ec1c0f927c542ef"} Jul 14 10:10:57.993872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.993852 2569 scope.go:117] "RemoveContainer" containerID="85808a59c8d1fdbd06ce6bb65759ebd2de7219db8759aa4e1ec1c0f927c542ef" Jul 14 10:10:57.994814 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.994789 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:10:57.994871 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.994854 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:10:57.994906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:57.994884 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:10:59.003666 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:59.003621 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" event={"ID":"fdce0995-348a-4885-825f-b040a0f4001c","Type":"ContainerStarted","Data":"55e9b8c95d439ed108e3dc56728ef1b95a8bf3348ec76f25c362e71f0743dc28"} Jul 14 10:10:59.477125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:59.477088 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:10:59.477371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:59.477344 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-btd4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 10:10:59.477425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:10:59.477402 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" podUID="fdce0995-348a-4885-825f-b040a0f4001c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 10:11:00.477758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:00.477719 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-btd4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 10:11:00.478139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:00.477776 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" podUID="fdce0995-348a-4885-825f-b040a0f4001c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 10:11:01.478308 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:01.478266 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-btd4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 10:11:01.478770 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:01.478330 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" podUID="fdce0995-348a-4885-825f-b040a0f4001c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 10:11:02.478240 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:02.478203 2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-btd4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 14 10:11:02.478411 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:02.478265 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" podUID="fdce0995-348a-4885-825f-b040a0f4001c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 14 10:11:03.481672 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:03.481640 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:11:03.482027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:03.481891 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:11:03.482714 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:03.482699 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-btd4c" Jul 14 10:11:14.554599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.554567 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s"] Jul 14 10:11:14.554960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.554935 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c6ea95-e3f6-458d-b205-3dea01c7e479" containerName="manager" Jul 14 10:11:14.554960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.554946 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c6ea95-e3f6-458d-b205-3dea01c7e479" containerName="manager" Jul 14 10:11:14.555029 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.555003 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c6ea95-e3f6-458d-b205-3dea01c7e479" containerName="manager" Jul 14 10:11:14.556943 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.556927 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.559442 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.559418 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 10:11:14.560791 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.560772 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j6fwv\"" Jul 14 10:11:14.560873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.560793 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 10:11:14.560873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.560817 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 10:11:14.568188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.568147 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s"] Jul 14 10:11:14.637304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637267 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khnl\" (UniqueName: \"kubernetes.io/projected/53c913f9-1218-4917-ae27-c2b9e876a501-kube-api-access-5khnl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.637304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637311 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.637528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637329 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c913f9-1218-4917-ae27-c2b9e876a501-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.637528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.637528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.637528 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.637528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738542 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738645 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khnl\" (UniqueName: \"kubernetes.io/projected/53c913f9-1218-4917-ae27-c2b9e876a501-kube-api-access-5khnl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738686 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c913f9-1218-4917-ae27-c2b9e876a501-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.738743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.738737 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.739030 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.739005 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.739092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.739026 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.739190 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.739102 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.741122 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.741096 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c913f9-1218-4917-ae27-c2b9e876a501-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.741367 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.741347 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c913f9-1218-4917-ae27-c2b9e876a501-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.746824 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.746801 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khnl\" (UniqueName: \"kubernetes.io/projected/53c913f9-1218-4917-ae27-c2b9e876a501-kube-api-access-5khnl\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xl9s\" (UID: \"53c913f9-1218-4917-ae27-c2b9e876a501\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:14.868208 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:14.868106 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:15.001296 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:15.001267 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s"] Jul 14 10:11:15.003840 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:11:15.003807 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c913f9_1218_4917_ae27_c2b9e876a501.slice/crio-50aad458cf045855d65c920a4fcd8c4bf8541fad8086957ad127fe9366f5deae WatchSource:0}: Error finding container 50aad458cf045855d65c920a4fcd8c4bf8541fad8086957ad127fe9366f5deae: Status 404 returned error can't find the container with id 50aad458cf045855d65c920a4fcd8c4bf8541fad8086957ad127fe9366f5deae Jul 14 10:11:15.005250 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:15.005233 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 10:11:15.102113 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:15.102079 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" event={"ID":"53c913f9-1218-4917-ae27-c2b9e876a501","Type":"ContainerStarted","Data":"50aad458cf045855d65c920a4fcd8c4bf8541fad8086957ad127fe9366f5deae"} Jul 14 10:11:17.113547 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:17.113508 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" event={"ID":"53c913f9-1218-4917-ae27-c2b9e876a501","Type":"ContainerStarted","Data":"3dff34c2be9cf80eba5efffaa1851e49069c634e27cf018a58bec5b49b196fa3"} Jul 14 10:11:17.137501 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:17.137433 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" podStartSLOduration=2.047034385 podStartE2EDuration="3.137417421s" totalImagesPullingTime="1.090383036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:11:17.134814312 +0000 UTC m=+661.768878153" watchObservedRunningTime="2026-07-14 10:11:17.137417421 +0000 UTC m=+661.771481261" Jul 14 10:11:22.154477 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.154440 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4"] Jul 14 10:11:22.157767 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.157747 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.160328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.160308 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 10:11:22.174612 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.174581 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4"] Jul 14 10:11:22.210511 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210469 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.210511 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210504 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.210697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210574 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.210697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210603 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p85rp\" (UniqueName: \"kubernetes.io/projected/812a40ed-eb22-4af2-895a-a443f134d687-kube-api-access-p85rp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.210697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210683 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a40ed-eb22-4af2-895a-a443f134d687-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.210798 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.210741 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311219 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311132 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p85rp\" (UniqueName: \"kubernetes.io/projected/812a40ed-eb22-4af2-895a-a443f134d687-kube-api-access-p85rp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311246 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a40ed-eb22-4af2-895a-a443f134d687-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311351 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311646 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311832 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311719 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.311832 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.311825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.313665 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.313644 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a40ed-eb22-4af2-895a-a443f134d687-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.313911 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.313891 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a40ed-eb22-4af2-895a-a443f134d687-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.320938 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.320918 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p85rp\" (UniqueName: \"kubernetes.io/projected/812a40ed-eb22-4af2-895a-a443f134d687-kube-api-access-p85rp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-htcx4\" (UID: \"812a40ed-eb22-4af2-895a-a443f134d687\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.467765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.467720 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:22.600641 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:22.600610 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4"] Jul 14 10:11:22.602398 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:11:22.602370 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812a40ed_eb22_4af2_895a_a443f134d687.slice/crio-633d2570284cc554913ae027619eb748e8b4c7b67c6d4df4dfc70f9c02c1b600 WatchSource:0}: Error finding container 633d2570284cc554913ae027619eb748e8b4c7b67c6d4df4dfc70f9c02c1b600: Status 404 returned error can't find the container with id 633d2570284cc554913ae027619eb748e8b4c7b67c6d4df4dfc70f9c02c1b600 Jul 14 10:11:23.146816 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:23.146725 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" event={"ID":"812a40ed-eb22-4af2-895a-a443f134d687","Type":"ContainerStarted","Data":"1035a93b91105f74d9b8005baf0c765f38618bc9d15518538d169a7c020b5ce1"} Jul 14 10:11:23.146816 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:23.146780 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" event={"ID":"812a40ed-eb22-4af2-895a-a443f134d687","Type":"ContainerStarted","Data":"633d2570284cc554913ae027619eb748e8b4c7b67c6d4df4dfc70f9c02c1b600"} Jul 14 10:11:23.170255 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:23.170203 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" podStartSLOduration=0.94546525 podStartE2EDuration="1.170184965s" totalImagesPullingTime="224.719715ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:11:23.167026467 +0000 UTC m=+667.801090320" watchObservedRunningTime="2026-07-14 10:11:23.170184965 +0000 UTC m=+667.804248805" Jul 14 10:11:24.868807 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:24.868763 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:24.869302 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:24.868941 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:24.881377 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:24.881354 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:25.167697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:25.167605 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xl9s" Jul 14 10:11:32.468734 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:32.468705 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:32.469245 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:32.468781 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:32.480970 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:32.480949 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:11:33.208905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:11:33.208879 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-htcx4" Jul 14 10:14:27.795650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:27.795614 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:14:27.796187 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:27.795817 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" podUID="f8156d66-50bb-4c71-be35-0c90f2be1003" containerName="manager" containerID="cri-o://285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097" gracePeriod=30 Jul 14 10:14:28.044729 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.044703 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:14:28.132768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.132666 2569 generic.go:356] "Generic (PLEG): container finished" podID="f8156d66-50bb-4c71-be35-0c90f2be1003" containerID="285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097" exitCode=0 Jul 14 10:14:28.132768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.132713 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" Jul 14 10:14:28.132768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.132748 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" event={"ID":"f8156d66-50bb-4c71-be35-0c90f2be1003","Type":"ContainerDied","Data":"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097"} Jul 14 10:14:28.132993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.132803 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-v2n2c" event={"ID":"f8156d66-50bb-4c71-be35-0c90f2be1003","Type":"ContainerDied","Data":"3092516bdbca5da7ddfdcf32b1e64f930cdbde9bb061e74ad6c8041835d04f3a"} Jul 14 10:14:28.132993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.132828 2569 scope.go:117] "RemoveContainer" containerID="285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097" Jul 14 10:14:28.141518 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.141500 2569 scope.go:117] "RemoveContainer" containerID="285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097" Jul 14 10:14:28.141789 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:14:28.141768 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097\": container with ID starting with 285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097 not found: ID does not exist" containerID="285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097" Jul 14 10:14:28.141863 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.141797 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097"} err="failed to get container status \"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097\": rpc error: code = NotFound desc = could not find container \"285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097\": container with ID starting with 285698f5ec2365f04bca3f06663a3eb7d2eb51ce1eb180a1e904457c50253097 not found: ID does not exist" Jul 14 10:14:28.150092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.150072 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert\") pod \"f8156d66-50bb-4c71-be35-0c90f2be1003\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " Jul 14 10:14:28.150223 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.150205 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdjdj\" (UniqueName: \"kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj\") pod \"f8156d66-50bb-4c71-be35-0c90f2be1003\" (UID: \"f8156d66-50bb-4c71-be35-0c90f2be1003\") " Jul 14 10:14:28.152199 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.152172 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f8156d66-50bb-4c71-be35-0c90f2be1003" (UID: "f8156d66-50bb-4c71-be35-0c90f2be1003"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 10:14:28.152321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.152302 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj" (OuterVolumeSpecName: "kube-api-access-bdjdj") pod "f8156d66-50bb-4c71-be35-0c90f2be1003" (UID: "f8156d66-50bb-4c71-be35-0c90f2be1003"). InnerVolumeSpecName "kube-api-access-bdjdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:14:28.251584 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.251557 2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8156d66-50bb-4c71-be35-0c90f2be1003-webhook-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:14:28.251584 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.251581 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdjdj\" (UniqueName: \"kubernetes.io/projected/f8156d66-50bb-4c71-be35-0c90f2be1003-kube-api-access-bdjdj\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:14:28.455519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.455489 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:14:28.459256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:28.459233 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-v2n2c"] Jul 14 10:14:29.188321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.188293 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-667d7fccf6-k9rsj"] Jul 14 10:14:29.188690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.188675 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8156d66-50bb-4c71-be35-0c90f2be1003" containerName="manager" Jul 14 10:14:29.188690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.188688 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8156d66-50bb-4c71-be35-0c90f2be1003" containerName="manager" Jul 14 10:14:29.188762 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.188746 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8156d66-50bb-4c71-be35-0c90f2be1003" containerName="manager" Jul 14 10:14:29.192778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.192761 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.195577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.195546 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 10:14:29.195717 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.195648 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 10:14:29.196856 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.196836 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z8s6p\"" Jul 14 10:14:29.202181 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.202162 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-k9rsj"] Jul 14 10:14:29.260315 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.260291 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b567d770-0548-448a-bdc1-4b03840a7869-webhook-cert\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.260424 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.260343 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xprqx\" (UniqueName: \"kubernetes.io/projected/b567d770-0548-448a-bdc1-4b03840a7869-kube-api-access-xprqx\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.361687 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.361662 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b567d770-0548-448a-bdc1-4b03840a7869-webhook-cert\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.361775 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.361715 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xprqx\" (UniqueName: \"kubernetes.io/projected/b567d770-0548-448a-bdc1-4b03840a7869-kube-api-access-xprqx\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.363982 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.363961 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b567d770-0548-448a-bdc1-4b03840a7869-webhook-cert\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.369195 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.369177 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xprqx\" (UniqueName: \"kubernetes.io/projected/b567d770-0548-448a-bdc1-4b03840a7869-kube-api-access-xprqx\") pod \"maas-controller-667d7fccf6-k9rsj\" (UID: \"b567d770-0548-448a-bdc1-4b03840a7869\") " pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.503715 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.503640 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:29.639406 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.639378 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-667d7fccf6-k9rsj"] Jul 14 10:14:29.640556 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:14:29.640522 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb567d770_0548_448a_bdc1_4b03840a7869.slice/crio-286bf0f1e720f582fd29ff1340f6a034b1d069877f2fa747c424f9b54e44b5bc WatchSource:0}: Error finding container 286bf0f1e720f582fd29ff1340f6a034b1d069877f2fa747c424f9b54e44b5bc: Status 404 returned error can't find the container with id 286bf0f1e720f582fd29ff1340f6a034b1d069877f2fa747c424f9b54e44b5bc Jul 14 10:14:29.870813 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:29.870744 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8156d66-50bb-4c71-be35-0c90f2be1003" path="/var/lib/kubelet/pods/f8156d66-50bb-4c71-be35-0c90f2be1003/volumes" Jul 14 10:14:30.151262 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:30.151174 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" event={"ID":"b567d770-0548-448a-bdc1-4b03840a7869","Type":"ContainerStarted","Data":"286bf0f1e720f582fd29ff1340f6a034b1d069877f2fa747c424f9b54e44b5bc"} Jul 14 10:14:31.157454 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:31.157408 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" event={"ID":"b567d770-0548-448a-bdc1-4b03840a7869","Type":"ContainerStarted","Data":"dd4e62b79b6a0934bca940a423ab458f2b56f24d03a91bfc0f9834c93e7e92b9"} Jul 14 10:14:31.157833 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:31.157539 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:14:31.176869 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:31.176828 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" podStartSLOduration=1.512938181 podStartE2EDuration="2.176816254s" totalImagesPullingTime="663.878073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:14:31.175389469 +0000 UTC m=+855.809453312" watchObservedRunningTime="2026-07-14 10:14:31.176816254 +0000 UTC m=+855.810880094" Jul 14 10:14:42.168494 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:14:42.168459 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-667d7fccf6-k9rsj" Jul 14 10:15:00.155294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.155261 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29733735-k6mfs"] Jul 14 10:15:00.158837 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.158818 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:00.162644 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.162627 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8pvgw\"" Jul 14 10:15:00.176239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.176217 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29733735-k6mfs"] Jul 14 10:15:00.315415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.315389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbb4t\" (UniqueName: \"kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t\") pod \"maas-api-key-cleanup-29733735-k6mfs\" (UID: \"84f9229e-545c-48be-a7e9-ed5620b55bf7\") " pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:00.416797 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.416727 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbb4t\" (UniqueName: \"kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t\") pod \"maas-api-key-cleanup-29733735-k6mfs\" (UID: \"84f9229e-545c-48be-a7e9-ed5620b55bf7\") " pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:00.435809 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.435787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbb4t\" (UniqueName: \"kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t\") pod \"maas-api-key-cleanup-29733735-k6mfs\" (UID: \"84f9229e-545c-48be-a7e9-ed5620b55bf7\") " pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:00.471519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.471498 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:00.610599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:00.610567 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29733735-k6mfs"] Jul 14 10:15:00.615445 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:15:00.615418 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f9229e_545c_48be_a7e9_ed5620b55bf7.slice/crio-3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712 WatchSource:0}: Error finding container 3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712: Status 404 returned error can't find the container with id 3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712 Jul 14 10:15:01.316227 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:01.316190 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" event={"ID":"84f9229e-545c-48be-a7e9-ed5620b55bf7","Type":"ContainerStarted","Data":"3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712"} Jul 14 10:15:03.328936 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:03.328898 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" event={"ID":"84f9229e-545c-48be-a7e9-ed5620b55bf7","Type":"ContainerStarted","Data":"50c944edc701ffc0576f8b7f74ef5737832d6b93f8a1121835984aa1da2a15e4"} Jul 14 10:15:04.335490 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:04.335455 2569 generic.go:356] "Generic (PLEG): container finished" podID="84f9229e-545c-48be-a7e9-ed5620b55bf7" containerID="50c944edc701ffc0576f8b7f74ef5737832d6b93f8a1121835984aa1da2a15e4" exitCode=0 Jul 14 10:15:04.335904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:04.335517 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" event={"ID":"84f9229e-545c-48be-a7e9-ed5620b55bf7","Type":"ContainerDied","Data":"50c944edc701ffc0576f8b7f74ef5737832d6b93f8a1121835984aa1da2a15e4"} Jul 14 10:15:05.477509 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:05.477487 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:05.565478 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:05.565452 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbb4t\" (UniqueName: \"kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t\") pod \"84f9229e-545c-48be-a7e9-ed5620b55bf7\" (UID: \"84f9229e-545c-48be-a7e9-ed5620b55bf7\") " Jul 14 10:15:05.567396 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:05.567367 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t" (OuterVolumeSpecName: "kube-api-access-sbb4t") pod "84f9229e-545c-48be-a7e9-ed5620b55bf7" (UID: "84f9229e-545c-48be-a7e9-ed5620b55bf7"). InnerVolumeSpecName "kube-api-access-sbb4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:15:05.666094 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:05.666041 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbb4t\" (UniqueName: \"kubernetes.io/projected/84f9229e-545c-48be-a7e9-ed5620b55bf7-kube-api-access-sbb4t\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:15:06.351850 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:06.351826 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" Jul 14 10:15:06.352005 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:06.351870 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733735-k6mfs" event={"ID":"84f9229e-545c-48be-a7e9-ed5620b55bf7","Type":"ContainerDied","Data":"3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712"} Jul 14 10:15:06.352005 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:06.351915 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aa6f36e4c1e1ad300390b880771f8c24b9e5f592abf4697e85ccf592ecf5712" Jul 14 10:15:15.869565 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:15.869492 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:15:15.869565 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:15:15.869492 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:20:15.906174 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:20:15.906135 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:20:15.908552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:20:15.908534 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:25:15.942615 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:25:15.942582 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:25:15.945884 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:25:15.945867 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:26:17.433388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.433345 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:17.434035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.434011 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f9229e-545c-48be-a7e9-ed5620b55bf7" containerName="cleanup" Jul 14 10:26:17.434194 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.434043 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f9229e-545c-48be-a7e9-ed5620b55bf7" containerName="cleanup" Jul 14 10:26:17.434194 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.434140 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="84f9229e-545c-48be-a7e9-ed5620b55bf7" containerName="cleanup" Jul 14 10:26:17.437605 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.437576 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.440324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.440297 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-62d45e95-openshift-default-dockercfg-488mq\"" Jul 14 10:26:17.457656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.457632 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:17.583511 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583473 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583511 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583509 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583551 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583666 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583715 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583757 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583798 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcc8f\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583842 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.583962 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.583874 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685093 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685071 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685116 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685139 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685199 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcc8f\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685235 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685264 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685260 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685297 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685542 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685586 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685668 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.685902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.685853 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.686045 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.686025 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.687596 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.687576 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.687707 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.687675 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.693363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.693338 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.693583 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.693565 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcc8f\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f\") pod \"e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.747941 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.747921 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:17.884048 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.884020 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:17.884713 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:26:17.884691 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfc2c25_8381_4c19_a9b2_6ce46cf2a2c7.slice/crio-d706b891cfe1a4792ed1996d5afd66f99158a28cc361401496fb18fbee506db2 WatchSource:0}: Error finding container d706b891cfe1a4792ed1996d5afd66f99158a28cc361401496fb18fbee506db2: Status 404 returned error can't find the container with id d706b891cfe1a4792ed1996d5afd66f99158a28cc361401496fb18fbee506db2 Jul 14 10:26:17.886842 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.886808 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:17.886954 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.886870 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:17.886954 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.886898 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:17.986556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.986524 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" event={"ID":"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7","Type":"ContainerStarted","Data":"25a1ff2976ecd59b3217d6c6168300d85298544a4a1d12356fd6bc39f4650a2d"} Jul 14 10:26:17.986723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:17.986575 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" event={"ID":"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7","Type":"ContainerStarted","Data":"d706b891cfe1a4792ed1996d5afd66f99158a28cc361401496fb18fbee506db2"} Jul 14 10:26:18.010120 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:18.010059 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" podStartSLOduration=1.010039781 podStartE2EDuration="1.010039781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:26:18.007663459 +0000 UTC m=+1562.641727301" watchObservedRunningTime="2026-07-14 10:26:18.010039781 +0000 UTC m=+1562.644103622" Jul 14 10:26:18.749104 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:18.749075 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:18.778906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:18.778864 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:18.991726 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:18.991692 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:18.992704 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:18.992685 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:23.843724 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:23.843689 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:23.844222 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:23.843925 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" podUID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" containerName="istio-proxy" containerID="cri-o://25a1ff2976ecd59b3217d6c6168300d85298544a4a1d12356fd6bc39f4650a2d" gracePeriod=30 Jul 14 10:26:29.046762 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.046732 2569 generic.go:356] "Generic (PLEG): container finished" podID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" containerID="25a1ff2976ecd59b3217d6c6168300d85298544a4a1d12356fd6bc39f4650a2d" exitCode=0 Jul 14 10:26:29.047080 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.046811 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" event={"ID":"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7","Type":"ContainerDied","Data":"25a1ff2976ecd59b3217d6c6168300d85298544a4a1d12356fd6bc39f4650a2d"} Jul 14 10:26:29.089700 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.089679 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:29.283422 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283386 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283450 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283477 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283498 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283556 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283550 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283583 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283633 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283659 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcc8f\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283678 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:29.283933 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283775 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert\") pod \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\" (UID: \"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7\") " Jul 14 10:26:29.283933 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283778 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:29.283933 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283788 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:29.283933 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.283896 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:29.284178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.284038 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.284178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.284055 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.284178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.284065 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.284178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.284082 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.284178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.284119 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:26:29.285918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.285891 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f" (OuterVolumeSpecName: "kube-api-access-kcc8f") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "kube-api-access-kcc8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:26:29.286028 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.285915 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:26:29.286076 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.286034 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:29.286076 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.286031 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" (UID: "5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:26:29.384422 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.384401 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.384523 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.384423 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcc8f\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-kube-api-access-kcc8f\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.384523 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.384439 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.384523 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.384451 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:29.384523 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:29.384462 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:30.053702 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.053672 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" Jul 14 10:26:30.053702 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.053686 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl" event={"ID":"5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7","Type":"ContainerDied","Data":"d706b891cfe1a4792ed1996d5afd66f99158a28cc361401496fb18fbee506db2"} Jul 14 10:26:30.054116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.053745 2569 scope.go:117] "RemoveContainer" containerID="25a1ff2976ecd59b3217d6c6168300d85298544a4a1d12356fd6bc39f4650a2d" Jul 14 10:26:30.076873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.076849 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:30.081576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.081556 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-62d45e95-openshift-default-cdfc588c9-gv7dl"] Jul 14 10:26:30.915471 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.915430 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:30.916058 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.916034 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" containerName="istio-proxy" Jul 14 10:26:30.916058 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.916058 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" containerName="istio-proxy" Jul 14 10:26:30.916247 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.916162 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" containerName="istio-proxy" Jul 14 10:26:30.926885 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.926860 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:30.929756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.929733 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f5f99a2e-openshift-default-dockercfg-mb7zq\"" Jul 14 10:26:30.933080 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:30.933056 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:31.097576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097541 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.097576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097578 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097619 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097711 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cls49\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097901 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.098059 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.097918 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198533 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198499 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198533 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198558 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198665 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198756 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198766 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198825 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198948 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198877 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.198948 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198913 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cls49\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.199026 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.198946 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.199076 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.199045 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.199197 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.199176 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.199299 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.199284 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.199349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.199323 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.201221 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.201203 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.201453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.201437 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.207526 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.207505 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.207686 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.207667 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cls49\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49\") pod \"e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.239289 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.239263 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:31.378362 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.378335 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:31.379676 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:26:31.379651 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37203232_b879_4d31_9c8f_4cdc3aa81afb.slice/crio-5b9507602e5c2efef4b37ef2f3c523dc06cb70d5e80aef1c6b967390de5089ad WatchSource:0}: Error finding container 5b9507602e5c2efef4b37ef2f3c523dc06cb70d5e80aef1c6b967390de5089ad: Status 404 returned error can't find the container with id 5b9507602e5c2efef4b37ef2f3c523dc06cb70d5e80aef1c6b967390de5089ad Jul 14 10:26:31.382145 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.382117 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:31.382241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.382194 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:31.382241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.382224 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:31.871903 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:31.871871 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7" path="/var/lib/kubelet/pods/5bfc2c25-8381-4c19-a9b2-6ce46cf2a2c7/volumes" Jul 14 10:26:32.067724 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:32.067690 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerStarted","Data":"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de"} Jul 14 10:26:32.067877 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:32.067744 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerStarted","Data":"5b9507602e5c2efef4b37ef2f3c523dc06cb70d5e80aef1c6b967390de5089ad"} Jul 14 10:26:32.098606 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:32.098543 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" podStartSLOduration=2.0985254700000002 podStartE2EDuration="2.09852547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:26:32.096086932 +0000 UTC m=+1576.730150776" watchObservedRunningTime="2026-07-14 10:26:32.09852547 +0000 UTC m=+1576.732589311" Jul 14 10:26:32.240121 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:32.240085 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:32.265105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:32.265071 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:33.073051 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:33.073016 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:33.073984 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:33.073961 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:43.136507 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.136475 2569 generic.go:356] "Generic (PLEG): container finished" podID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerID="d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de" exitCode=0 Jul 14 10:26:43.136913 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.136554 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerDied","Data":"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de"} Jul 14 10:26:43.136961 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.136914 2569 scope.go:117] "RemoveContainer" containerID="d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de" Jul 14 10:26:43.137825 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.137796 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:43.137927 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.137877 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:43.137927 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.137921 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:43.182204 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:43.182178 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:44.143559 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:44.143526 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerStarted","Data":"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a"} Jul 14 10:26:44.143971 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:44.143604 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" containerID="cri-o://ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a" gracePeriod=30 Jul 14 10:26:48.073253 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:48.073219 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:49.403437 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.403385 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:49.462428 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462403 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462435 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462482 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462502 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cls49\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462553 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462520 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462561 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462587 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462612 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462642 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data\") pod \"37203232-b879-4d31-9c8f-4cdc3aa81afb\" (UID: \"37203232-b879-4d31-9c8f-4cdc3aa81afb\") " Jul 14 10:26:49.462778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462664 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:49.463043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462806 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:49.463043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.462866 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:49.463043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.463016 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.463043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.463036 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.463305 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.463050 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.463305 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.463012 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:26:49.463886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.463859 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data" (OuterVolumeSpecName: "istio-data") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:49.464886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.464862 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:26:49.465026 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.465001 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:26:49.465106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.465065 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49" (OuterVolumeSpecName: "kube-api-access-cls49") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "kube-api-access-cls49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:26:49.465518 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.465499 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token" (OuterVolumeSpecName: "istio-token") pod "37203232-b879-4d31-9c8f-4cdc3aa81afb" (UID: "37203232-b879-4d31-9c8f-4cdc3aa81afb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:26:49.563589 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563558 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.563589 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563588 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cls49\" (UniqueName: \"kubernetes.io/projected/37203232-b879-4d31-9c8f-4cdc3aa81afb-kube-api-access-cls49\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.563589 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563597 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37203232-b879-4d31-9c8f-4cdc3aa81afb-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.563750 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563607 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.563750 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563614 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:49.563750 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:49.563621 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37203232-b879-4d31-9c8f-4cdc3aa81afb-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:26:50.177881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.177808 2569 generic.go:356] "Generic (PLEG): container finished" podID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerID="ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a" exitCode=0 Jul 14 10:26:50.177881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.177853 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" Jul 14 10:26:50.178056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.177884 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerDied","Data":"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a"} Jul 14 10:26:50.178056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.177935 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m" event={"ID":"37203232-b879-4d31-9c8f-4cdc3aa81afb","Type":"ContainerDied","Data":"5b9507602e5c2efef4b37ef2f3c523dc06cb70d5e80aef1c6b967390de5089ad"} Jul 14 10:26:50.178056 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.177957 2569 scope.go:117] "RemoveContainer" containerID="ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a" Jul 14 10:26:50.186972 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.186955 2569 scope.go:117] "RemoveContainer" containerID="d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de" Jul 14 10:26:50.195017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.195002 2569 scope.go:117] "RemoveContainer" containerID="ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a" Jul 14 10:26:50.195281 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:26:50.195264 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a\": container with ID starting with ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a not found: ID does not exist" containerID="ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a" Jul 14 10:26:50.195348 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.195294 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a"} err="failed to get container status \"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a\": rpc error: code = NotFound desc = could not find container \"ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a\": container with ID starting with ac3661b3194ce2e9771e0c24367e6687e52c11fd8015b6415d6fbcb6bba1c27a not found: ID does not exist" Jul 14 10:26:50.195348 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.195310 2569 scope.go:117] "RemoveContainer" containerID="d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de" Jul 14 10:26:50.195523 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:26:50.195505 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de\": container with ID starting with d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de not found: ID does not exist" containerID="d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de" Jul 14 10:26:50.195569 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.195529 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de"} err="failed to get container status \"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de\": rpc error: code = NotFound desc = could not find container \"d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de\": container with ID starting with d08393d8dac6237de8b2cb9a853e706248fd936766b85ba634213b81be3b62de not found: ID does not exist" Jul 14 10:26:50.203124 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.203099 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:50.205852 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.205832 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f5f99a2e-openshift-default-f575b7957-glv8m"] Jul 14 10:26:50.476202 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.476170 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:26:50.476857 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.476832 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:26:50.476982 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.476924 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:26:50.477102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.477087 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:26:50.477270 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.477256 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:26:50.482045 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.482025 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.484906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.484883 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1060a842-openshift-default-dockercfg-xgvbt\"" Jul 14 10:26:50.497070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.497047 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:26:50.570660 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570633 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570675 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570691 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570744 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570785 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.570916 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.571042 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.570960 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.571042 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.571013 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bf96\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.671836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.671799 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.671836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.671839 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bf96\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.671867 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.671887 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.671902 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672018 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672070 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672060 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672361 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672084 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672361 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672410 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672450 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672498 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672703 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672557 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.672759 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.672714 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.674143 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.674120 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.674457 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.674442 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.681279 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.681254 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.681513 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.681497 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bf96\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96\") pod \"e2e-derive-1060a842-openshift-default-69cf699955-mqgtq\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.793926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.793872 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:50.939989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.939919 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:26:50.942307 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:26:50.942281 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07f0b78_d1d0_4ff6_9d06_54e14ecc24c7.slice/crio-2d56c96862237247acf0b42e66132359556cc5e76223c4829b22e17ab2f3c835 WatchSource:0}: Error finding container 2d56c96862237247acf0b42e66132359556cc5e76223c4829b22e17ab2f3c835: Status 404 returned error can't find the container with id 2d56c96862237247acf0b42e66132359556cc5e76223c4829b22e17ab2f3c835 Jul 14 10:26:50.944442 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.944410 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:50.944520 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.944478 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:50.944520 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:50.944505 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:26:51.187926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:51.187888 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" event={"ID":"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7","Type":"ContainerStarted","Data":"1da3970883f00fa64618aeaf0929137ca3543efc08615fd6b3257204564d6221"} Jul 14 10:26:51.188094 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:51.187946 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" event={"ID":"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7","Type":"ContainerStarted","Data":"2d56c96862237247acf0b42e66132359556cc5e76223c4829b22e17ab2f3c835"} Jul 14 10:26:51.217644 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:51.217600 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" podStartSLOduration=1.217577398 podStartE2EDuration="1.217577398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:26:51.216617063 +0000 UTC m=+1595.850680907" watchObservedRunningTime="2026-07-14 10:26:51.217577398 +0000 UTC m=+1595.851641238" Jul 14 10:26:51.794760 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:51.794730 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:51.878068 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:51.878034 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" path="/var/lib/kubelet/pods/37203232-b879-4d31-9c8f-4cdc3aa81afb/volumes" Jul 14 10:26:52.799392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:52.799363 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:53.200215 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:53.200183 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:53.201195 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:53.201175 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:26:56.270279 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:56.270244 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:26:56.270628 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:26:56.270426 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" podUID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" containerName="istio-proxy" containerID="cri-o://1da3970883f00fa64618aeaf0929137ca3543efc08615fd6b3257204564d6221" gracePeriod=30 Jul 14 10:27:02.263380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.263310 2569 generic.go:356] "Generic (PLEG): container finished" podID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" containerID="1da3970883f00fa64618aeaf0929137ca3543efc08615fd6b3257204564d6221" exitCode=0 Jul 14 10:27:02.263380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.263352 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" event={"ID":"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7","Type":"ContainerDied","Data":"1da3970883f00fa64618aeaf0929137ca3543efc08615fd6b3257204564d6221"} Jul 14 10:27:02.319814 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.319793 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:27:02.487050 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487025 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487050 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487054 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487114 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487139 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487175 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487193 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487207 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bf96\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487228 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487259 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert\") pod \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\" (UID: \"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7\") " Jul 14 10:27:02.487616 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487527 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:27:02.487684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487651 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:27:02.487771 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487751 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:27:02.487876 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.487855 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:27:02.488533 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.488503 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:27:02.489387 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.489362 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:27:02.489470 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.489425 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:27:02.489517 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.489469 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:27:02.489517 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.489482 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96" (OuterVolumeSpecName: "kube-api-access-2bf96") pod "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" (UID: "e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7"). InnerVolumeSpecName "kube-api-access-2bf96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:27:02.588751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588727 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588748 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588757 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588766 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588774 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bf96\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-kube-api-access-2bf96\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588782 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588790 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588797 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:02.588873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:02.588804 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:27:03.270483 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.270452 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" Jul 14 10:27:03.270483 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.270467 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq" event={"ID":"e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7","Type":"ContainerDied","Data":"2d56c96862237247acf0b42e66132359556cc5e76223c4829b22e17ab2f3c835"} Jul 14 10:27:03.270993 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.270521 2569 scope.go:117] "RemoveContainer" containerID="1da3970883f00fa64618aeaf0929137ca3543efc08615fd6b3257204564d6221" Jul 14 10:27:03.298415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.298392 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:27:03.304433 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.304413 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1060a842-openshift-default-69cf699955-mqgtq"] Jul 14 10:27:03.871173 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:27:03.871116 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" path="/var/lib/kubelet/pods/e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7/volumes" Jul 14 10:28:44.401802 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.401724 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:28:44.402191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.402115 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:28:44.402191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.402129 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="37203232-b879-4d31-9c8f-4cdc3aa81afb" containerName="istio-proxy" Jul 14 10:28:44.402191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.402144 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" containerName="istio-proxy" Jul 14 10:28:44.402191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.402172 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" containerName="istio-proxy" Jul 14 10:28:44.402328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.402244 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e07f0b78-d1d0-4ff6-9d06-54e14ecc24c7" containerName="istio-proxy" Jul 14 10:28:44.405343 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.405327 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.408189 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.408169 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-288cd506-openshift-default-dockercfg-r6mpv\"" Jul 14 10:28:44.428105 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.428081 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:28:44.514356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514324 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514379 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514459 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514510 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514532 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514553 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514601 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phxlf\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514629 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.514761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.514666 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.615986 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.615954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.615997 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phxlf\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616015 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616042 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616208 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616244 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616268 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616576 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616540 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616627 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616651 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616642 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616709 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.616765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.616733 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.618318 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.618291 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.618443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.618425 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.624448 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.624423 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.624677 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.624658 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phxlf\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf\") pod \"e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.715875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.715854 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:44.866974 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.866949 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:28:44.868937 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:28:44.868906 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9783dcb5_8360_4823_b606_29e171b8a905.slice/crio-81fa3285bae4dc1993e91f0864fa5288851937cccd2bda2d4c18c414d42c853e WatchSource:0}: Error finding container 81fa3285bae4dc1993e91f0864fa5288851937cccd2bda2d4c18c414d42c853e: Status 404 returned error can't find the container with id 81fa3285bae4dc1993e91f0864fa5288851937cccd2bda2d4c18c414d42c853e Jul 14 10:28:44.871431 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.871388 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:28:44.871533 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.871506 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:28:44.871577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:44.871554 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:28:45.813979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:45.813940 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerStarted","Data":"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5"} Jul 14 10:28:45.814467 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:45.814009 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerStarted","Data":"81fa3285bae4dc1993e91f0864fa5288851937cccd2bda2d4c18c414d42c853e"} Jul 14 10:28:45.842702 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:45.842648 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podStartSLOduration=1.842633759 podStartE2EDuration="1.842633759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:28:45.84038859 +0000 UTC m=+1710.474452432" watchObservedRunningTime="2026-07-14 10:28:45.842633759 +0000 UTC m=+1710.476697599" Jul 14 10:28:46.716525 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:46.716490 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:46.720773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:46.720746 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:46.819376 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:46.819354 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:28:46.819989 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:28:46.819972 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:02.464872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.464043 2569 patch_prober.go:28] interesting pod/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:44794->10.133.0.49:15021: read: connection reset by peer" start-of-body= Jul 14 10:29:02.464872 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.464121 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:44794->10.133.0.49:15021: read: connection reset by peer" Jul 14 10:29:02.906587 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.906517 2569 generic.go:356] "Generic (PLEG): container finished" podID="9783dcb5-8360-4823-b606-29e171b8a905" containerID="f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5" exitCode=0 Jul 14 10:29:02.906587 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.906573 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerDied","Data":"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5"} Jul 14 10:29:02.906956 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.906939 2569 scope.go:117] "RemoveContainer" containerID="f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5" Jul 14 10:29:02.908013 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.907986 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:02.908101 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.908082 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:02.908144 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:02.908127 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:03.921481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:03.921433 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerStarted","Data":"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d"} Jul 14 10:29:04.715990 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:04.715955 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:04.716638 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:04.716334 2569 patch_prober.go:28] interesting pod/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 10:29:04.716638 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:04.716394 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 10:29:05.717047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:05.716578 2569 patch_prober.go:28] interesting pod/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 10:29:05.717047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:05.716654 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 10:29:06.716250 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:06.716209 2569 patch_prober.go:28] interesting pod/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 10:29:06.716551 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:06.716523 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 10:29:07.716952 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:07.716913 2569 patch_prober.go:28] interesting pod/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 10:29:07.717567 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:07.717535 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 10:29:08.721087 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:08.721056 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:08.721513 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:08.721343 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:08.721991 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:08.721974 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:09.449868 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:09.449827 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:29:09.960726 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:09.960694 2569 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" secret="" err="secret \"e2e-mt-288cd506-openshift-default-dockercfg-r6mpv\" not found" Jul 14 10:29:10.965765 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:10.965725 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" containerID="cri-o://42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d" gracePeriod=30 Jul 14 10:29:16.217040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.217016 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:16.306207 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306104 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phxlf\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306207 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306168 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306207 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306185 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306227 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306255 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306291 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306320 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306352 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306378 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data\") pod \"9783dcb5-8360-4823-b606-29e171b8a905\" (UID: \"9783dcb5-8360-4823-b606-29e171b8a905\") " Jul 14 10:29:16.306776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306493 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:29:16.306776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306612 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:29:16.306776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306692 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:29:16.306941 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306891 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.306941 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306903 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:29:16.306941 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306910 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.307107 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.306944 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.307320 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.307293 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data" (OuterVolumeSpecName: "istio-data") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:29:16.308488 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.308452 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf" (OuterVolumeSpecName: "kube-api-access-phxlf") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "kube-api-access-phxlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:29:16.308605 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.308507 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:29:16.308605 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.308525 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:29:16.308740 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.308725 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token" (OuterVolumeSpecName: "istio-token") pod "9783dcb5-8360-4823-b606-29e171b8a905" (UID: "9783dcb5-8360-4823-b606-29e171b8a905"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:29:16.407983 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.407963 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.407983 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.407982 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9783dcb5-8360-4823-b606-29e171b8a905-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.408142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.407991 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.408142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.407999 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9783dcb5-8360-4823-b606-29e171b8a905-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.408142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.408008 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9783dcb5-8360-4823-b606-29e171b8a905-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.408142 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.408021 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phxlf\" (UniqueName: \"kubernetes.io/projected/9783dcb5-8360-4823-b606-29e171b8a905-kube-api-access-phxlf\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:29:16.998523 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.998488 2569 generic.go:356] "Generic (PLEG): container finished" podID="9783dcb5-8360-4823-b606-29e171b8a905" containerID="42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d" exitCode=0 Jul 14 10:29:16.998668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.998545 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" Jul 14 10:29:16.998668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.998580 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerDied","Data":"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d"} Jul 14 10:29:16.998668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.998633 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2" event={"ID":"9783dcb5-8360-4823-b606-29e171b8a905","Type":"ContainerDied","Data":"81fa3285bae4dc1993e91f0864fa5288851937cccd2bda2d4c18c414d42c853e"} Jul 14 10:29:16.998668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:16.998656 2569 scope.go:117] "RemoveContainer" containerID="42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d" Jul 14 10:29:17.007840 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.007823 2569 scope.go:117] "RemoveContainer" containerID="f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5" Jul 14 10:29:17.015690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.015675 2569 scope.go:117] "RemoveContainer" containerID="42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d" Jul 14 10:29:17.015909 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:29:17.015892 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d\": container with ID starting with 42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d not found: ID does not exist" containerID="42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d" Jul 14 10:29:17.015968 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.015914 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d"} err="failed to get container status \"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d\": rpc error: code = NotFound desc = could not find container \"42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d\": container with ID starting with 42f6cbd23a5840c7d97636b19d2d2ae864f25f12ab603a41f139753cb580915d not found: ID does not exist" Jul 14 10:29:17.015968 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.015929 2569 scope.go:117] "RemoveContainer" containerID="f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5" Jul 14 10:29:17.016165 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:29:17.016131 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5\": container with ID starting with f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5 not found: ID does not exist" containerID="f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5" Jul 14 10:29:17.016210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.016169 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5"} err="failed to get container status \"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5\": rpc error: code = NotFound desc = could not find container \"f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5\": container with ID starting with f59301a665024561696db40eda0d79f0b320282472300ef9ab5986d48ae846b5 not found: ID does not exist" Jul 14 10:29:17.035339 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.035316 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:29:17.046223 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.046203 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-288cd506-openshift-default-9f5bbccdc-lgrx2"] Jul 14 10:29:17.871169 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:17.871118 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9783dcb5-8360-4823-b606-29e171b8a905" path="/var/lib/kubelet/pods/9783dcb5-8360-4823-b606-29e171b8a905/volumes" Jul 14 10:29:30.918875 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.918844 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:29:30.919349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.919244 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:29:30.919349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.919258 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:29:30.919349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.919319 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:29:30.919349 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.919331 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:29:30.923697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.923679 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:30.926378 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.926357 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b18fcbaa-openshift-default-dockercfg-9rqm5\"" Jul 14 10:29:30.938702 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:30.938675 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:29:31.024751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024719 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pxbc\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.024751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024756 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.024997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.024997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.024997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024958 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.024997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.024989 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.025218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.025034 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.025218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.025094 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.025218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.025164 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.125940 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.125905 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.125957 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pxbc\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126146 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126124 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126146 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126144 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126255 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126183 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126253 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126310 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126295 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126472 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126323 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126552 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126858 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126584 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.126923 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.126897 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.127027 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.127010 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.128468 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.128449 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.128690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.128673 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.135309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.135285 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.135670 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.135654 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pxbc\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc\") pod \"e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.233858 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.233830 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:31.375401 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.375378 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:29:31.377091 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:29:31.377065 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8e131b_f9f8_41f4_9d61_1cadb54c4d4b.slice/crio-bdd16136fbcc80cd59f7bafc3bc2484af95c5fc8d3c6436f83e0fdbd85358467 WatchSource:0}: Error finding container bdd16136fbcc80cd59f7bafc3bc2484af95c5fc8d3c6436f83e0fdbd85358467: Status 404 returned error can't find the container with id bdd16136fbcc80cd59f7bafc3bc2484af95c5fc8d3c6436f83e0fdbd85358467 Jul 14 10:29:31.379248 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.379219 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:31.379321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.379279 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:31.379321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:31.379307 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:32.078929 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:32.078894 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerStarted","Data":"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316"} Jul 14 10:29:32.079334 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:32.078947 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerStarted","Data":"bdd16136fbcc80cd59f7bafc3bc2484af95c5fc8d3c6436f83e0fdbd85358467"} Jul 14 10:29:32.117874 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:32.117823 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podStartSLOduration=2.117808163 podStartE2EDuration="2.117808163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:29:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:29:32.114484465 +0000 UTC m=+1756.748548307" watchObservedRunningTime="2026-07-14 10:29:32.117808163 +0000 UTC m=+1756.751872004" Jul 14 10:29:32.234961 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:32.234931 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:32.239391 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:32.239363 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:33.084408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:33.084379 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:33.085309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:33.085287 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:48.230521 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:48.229058 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:38558->10.133.0.50:15021: read: connection reset by peer" start-of-body= Jul 14 10:29:48.230521 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:48.229133 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:38558->10.133.0.50:15021: read: connection reset by peer" Jul 14 10:29:49.179615 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.179579 2569 generic.go:356] "Generic (PLEG): container finished" podID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerID="300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316" exitCode=0 Jul 14 10:29:49.179798 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.179658 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerDied","Data":"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316"} Jul 14 10:29:49.180122 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.180107 2569 scope.go:117] "RemoveContainer" containerID="300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316" Jul 14 10:29:49.181139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.181112 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:49.181266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.181196 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:49.181266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:49.181229 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:29:50.187103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:50.187074 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerStarted","Data":"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166"} Jul 14 10:29:50.234192 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:50.234135 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:50.234388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:50.234362 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:29:50.234467 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:50.234413 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 10:29:51.234577 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:51.234538 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:29:51.235019 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:51.234595 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 10:29:52.234481 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:52.234446 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:29:52.234650 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:52.234505 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 10:29:53.234363 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:53.234323 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:29:53.234562 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:53.234382 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 10:29:54.234598 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:54.234556 2569 patch_prober.go:28] interesting pod/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 10:29:54.235083 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:54.234657 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 10:29:55.237535 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:55.237504 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:55.237969 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:55.237869 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:29:55.238563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:29:55.238544 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:30:00.140611 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.140583 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29733750-s5wr8"] Jul 14 10:30:00.140948 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.140918 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:30:00.140948 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.140930 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9783dcb5-8360-4823-b606-29e171b8a905" containerName="istio-proxy" Jul 14 10:30:00.144085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.144062 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:00.147102 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.147076 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-8pvgw\"" Jul 14 10:30:00.154062 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.154038 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29733750-s5wr8"] Jul 14 10:30:00.296935 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.296903 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tjgj\" (UniqueName: \"kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj\") pod \"maas-api-key-cleanup-29733750-s5wr8\" (UID: \"f1de1108-9f98-42fb-8ec1-07a9eabe2ace\") " pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:00.397736 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.397660 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tjgj\" (UniqueName: \"kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj\") pod \"maas-api-key-cleanup-29733750-s5wr8\" (UID: \"f1de1108-9f98-42fb-8ec1-07a9eabe2ace\") " pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:00.407761 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.407738 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tjgj\" (UniqueName: \"kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj\") pod \"maas-api-key-cleanup-29733750-s5wr8\" (UID: \"f1de1108-9f98-42fb-8ec1-07a9eabe2ace\") " pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:00.456656 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.456636 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:00.589898 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:00.589873 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29733750-s5wr8"] Jul 14 10:30:00.591892 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:30:00.591866 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1de1108_9f98_42fb_8ec1_07a9eabe2ace.slice/crio-5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53 WatchSource:0}: Error finding container 5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53: Status 404 returned error can't find the container with id 5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53 Jul 14 10:30:01.253024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:01.252991 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" event={"ID":"f1de1108-9f98-42fb-8ec1-07a9eabe2ace","Type":"ContainerStarted","Data":"0b0dc227a8f1ea52e148d7b1075b3d7800793f4587d9ea182c58f064fb2bc75b"} Jul 14 10:30:01.253394 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:01.253046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" event={"ID":"f1de1108-9f98-42fb-8ec1-07a9eabe2ace","Type":"ContainerStarted","Data":"5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53"} Jul 14 10:30:01.275988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:01.275941 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" podStartSLOduration=1.275926282 podStartE2EDuration="1.275926282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:30:01.274312401 +0000 UTC m=+1785.908376243" watchObservedRunningTime="2026-07-14 10:30:01.275926282 +0000 UTC m=+1785.909990122" Jul 14 10:30:02.260331 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:02.260298 2569 generic.go:356] "Generic (PLEG): container finished" podID="f1de1108-9f98-42fb-8ec1-07a9eabe2ace" containerID="0b0dc227a8f1ea52e148d7b1075b3d7800793f4587d9ea182c58f064fb2bc75b" exitCode=0 Jul 14 10:30:02.260739 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:02.260377 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" event={"ID":"f1de1108-9f98-42fb-8ec1-07a9eabe2ace","Type":"ContainerDied","Data":"0b0dc227a8f1ea52e148d7b1075b3d7800793f4587d9ea182c58f064fb2bc75b"} Jul 14 10:30:03.393277 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:03.393253 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:03.521629 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:03.521557 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tjgj\" (UniqueName: \"kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj\") pod \"f1de1108-9f98-42fb-8ec1-07a9eabe2ace\" (UID: \"f1de1108-9f98-42fb-8ec1-07a9eabe2ace\") " Jul 14 10:30:03.523513 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:03.523483 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj" (OuterVolumeSpecName: "kube-api-access-6tjgj") pod "f1de1108-9f98-42fb-8ec1-07a9eabe2ace" (UID: "f1de1108-9f98-42fb-8ec1-07a9eabe2ace"). InnerVolumeSpecName "kube-api-access-6tjgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:30:03.622555 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:03.622522 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tjgj\" (UniqueName: \"kubernetes.io/projected/f1de1108-9f98-42fb-8ec1-07a9eabe2ace-kube-api-access-6tjgj\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:30:04.271930 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:04.271901 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" Jul 14 10:30:04.272078 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:04.271944 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29733750-s5wr8" event={"ID":"f1de1108-9f98-42fb-8ec1-07a9eabe2ace","Type":"ContainerDied","Data":"5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53"} Jul 14 10:30:04.272078 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:04.271997 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a28df8abfc09d95c300c907917955f2b2f5fdefe678bff74b585322ebb1ca53" Jul 14 10:30:15.978092 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:15.978065 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:30:15.982667 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:30:15.982643 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:31:00.997583 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:00.997551 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:31:00.998143 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:00.997787 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" containerID="cri-o://5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166" gracePeriod=30 Jul 14 10:31:06.250820 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.250791 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:31:06.371077 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371009 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371077 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371071 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371108 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371131 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pxbc\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371174 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371226 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371296 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371328 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:31:06.371748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371451 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:31:06.371748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371531 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371602 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:31:06.371748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371620 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert\") pod \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\" (UID: \"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b\") " Jul 14 10:31:06.371960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371922 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.371960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371940 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.371960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371939 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:31:06.371960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.371955 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.372415 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.372384 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:31:06.373660 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.373630 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:31:06.373660 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.373628 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:31:06.373795 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.373724 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:31:06.373920 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.373902 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc" (OuterVolumeSpecName: "kube-api-access-5pxbc") pod "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" (UID: "cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b"). InnerVolumeSpecName "kube-api-access-5pxbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:31:06.472997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.472966 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.472997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.472991 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.472997 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.472999 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.473235 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.473008 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.473235 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.473018 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pxbc\" (UniqueName: \"kubernetes.io/projected/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-kube-api-access-5pxbc\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.473235 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.473026 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:31:06.614683 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.614642 2569 generic.go:356] "Generic (PLEG): container finished" podID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerID="5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166" exitCode=0 Jul 14 10:31:06.614870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.614708 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" Jul 14 10:31:06.614870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.614728 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerDied","Data":"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166"} Jul 14 10:31:06.614870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.614824 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz" event={"ID":"cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b","Type":"ContainerDied","Data":"bdd16136fbcc80cd59f7bafc3bc2484af95c5fc8d3c6436f83e0fdbd85358467"} Jul 14 10:31:06.615040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.614877 2569 scope.go:117] "RemoveContainer" containerID="5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166" Jul 14 10:31:06.625094 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.625075 2569 scope.go:117] "RemoveContainer" containerID="300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316" Jul 14 10:31:06.633200 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.633182 2569 scope.go:117] "RemoveContainer" containerID="5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166" Jul 14 10:31:06.633444 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:31:06.633426 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166\": container with ID starting with 5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166 not found: ID does not exist" containerID="5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166" Jul 14 10:31:06.633503 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.633451 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166"} err="failed to get container status \"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166\": rpc error: code = NotFound desc = could not find container \"5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166\": container with ID starting with 5baf9b5c3c7b9ad85977d1200c256019a5e92dfa0e4799d838e11540bd51e166 not found: ID does not exist" Jul 14 10:31:06.633503 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.633465 2569 scope.go:117] "RemoveContainer" containerID="300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316" Jul 14 10:31:06.633684 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:31:06.633663 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316\": container with ID starting with 300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316 not found: ID does not exist" containerID="300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316" Jul 14 10:31:06.633751 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.633694 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316"} err="failed to get container status \"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316\": rpc error: code = NotFound desc = could not find container \"300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316\": container with ID starting with 300dbbbaa4c9c1ae9f4c260d1b3be233e21abb4184dd115ed6784c68c25d7316 not found: ID does not exist" Jul 14 10:31:06.639081 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.639060 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:31:06.644233 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:06.644212 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b18fcbaa-openshift-default-5dd59bf4c5-fhshz"] Jul 14 10:31:07.871078 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:07.871040 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" path="/var/lib/kubelet/pods/cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b/volumes" Jul 14 10:31:56.583904 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.583829 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584236 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1de1108-9f98-42fb-8ec1-07a9eabe2ace" containerName="cleanup" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584248 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1de1108-9f98-42fb-8ec1-07a9eabe2ace" containerName="cleanup" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584269 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584274 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584283 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584289 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584346 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.584353 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584355 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1de1108-9f98-42fb-8ec1-07a9eabe2ace" containerName="cleanup" Jul 14 10:31:56.584604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.584360 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8e131b-f9f8-41f4-9d61-1cadb54c4d4b" containerName="istio-proxy" Jul 14 10:31:56.587317 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.587300 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.590044 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.590022 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-37ad9d-openshift-default-dockercfg-bv9wd\"" Jul 14 10:31:56.597901 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.597879 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:31:56.734778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734784 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734873 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734922 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734957 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6scb\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.734985 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.734983 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.735212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.735033 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.735212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.735097 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836478 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836392 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836478 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836437 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836705 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836620 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836705 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836685 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836820 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836732 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836820 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836785 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6scb\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836917 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836831 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.836917 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836885 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.836928 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837072 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.837040 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837143 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.837126 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837249 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.837227 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.837319 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.837414 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.837395 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.839127 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.839099 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.839496 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.839474 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.845912 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.845882 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6scb\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.846868 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.846847 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token\") pod \"e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:56.898605 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:56.898575 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:57.033055 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.033022 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:31:57.037935 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:31:57.037905 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6969c9_b9de_4cf7_91db_4b45a21af85b.slice/crio-7f73f10d9ec95a08e7fde4cd3e07097fbed85dea49597e1fe869cef8087e2355 WatchSource:0}: Error finding container 7f73f10d9ec95a08e7fde4cd3e07097fbed85dea49597e1fe869cef8087e2355: Status 404 returned error can't find the container with id 7f73f10d9ec95a08e7fde4cd3e07097fbed85dea49597e1fe869cef8087e2355 Jul 14 10:31:57.040321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.040290 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:31:57.040408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.040354 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:31:57.040408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.040382 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:31:57.893570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.893539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerStarted","Data":"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9"} Jul 14 10:31:57.893918 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.893584 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerStarted","Data":"7f73f10d9ec95a08e7fde4cd3e07097fbed85dea49597e1fe869cef8087e2355"} Jul 14 10:31:57.898842 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.898820 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:57.920655 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:57.920598 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podStartSLOduration=1.920583725 podStartE2EDuration="1.920583725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:31:57.917620631 +0000 UTC m=+1902.551684469" watchObservedRunningTime="2026-07-14 10:31:57.920583725 +0000 UTC m=+1902.554647653" Jul 14 10:31:58.904285 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:58.904253 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:59.905430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:59.905399 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:31:59.906269 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:31:59.906249 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:32:04.990682 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:04.990647 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:32:04.994260 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:04.994243 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:04.999253 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:04.999231 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1242c6-openshift-default-dockercfg-c4jnv\"" Jul 14 10:32:05.017550 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.017523 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:32:05.104742 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.104742 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104751 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.104992 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104843 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqrhg\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.104992 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104899 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.104992 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.104992 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.104948 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.105239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.105040 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.105239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.105063 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.105239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.105106 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206206 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206143 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206253 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206279 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqrhg\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206316 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206344 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206671 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206429 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206671 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206461 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206883 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.206946 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206903 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.207138 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206966 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.207138 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.206997 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.207138 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.207060 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.208564 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.208541 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.208943 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.208922 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.219984 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.219952 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.221178 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.221131 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqrhg\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg\") pod \"e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.305062 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.304992 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:05.444604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.444578 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:32:05.446533 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:32:05.446508 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85392f7e_d810_46ea_be30_64657fd89f9e.slice/crio-018f892672d668ce2eef055d12826ba96625b85d9268fc7c0545a657eb5ff86c WatchSource:0}: Error finding container 018f892672d668ce2eef055d12826ba96625b85d9268fc7c0545a657eb5ff86c: Status 404 returned error can't find the container with id 018f892672d668ce2eef055d12826ba96625b85d9268fc7c0545a657eb5ff86c Jul 14 10:32:05.448563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.448529 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:05.448684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.448610 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:05.448684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.448655 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:05.939947 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.939910 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerStarted","Data":"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd"} Jul 14 10:32:05.940131 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.939959 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerStarted","Data":"018f892672d668ce2eef055d12826ba96625b85d9268fc7c0545a657eb5ff86c"} Jul 14 10:32:05.966350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:05.966292 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podStartSLOduration=1.966272644 podStartE2EDuration="1.966272644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:32:05.96348775 +0000 UTC m=+1910.597551591" watchObservedRunningTime="2026-07-14 10:32:05.966272644 +0000 UTC m=+1910.600336485" Jul 14 10:32:06.305465 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:06.305391 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:06.310218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:06.310194 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:06.945520 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:06.945486 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:06.946456 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:06.946434 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:15.093684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:15.092378 2569 patch_prober.go:28] interesting pod/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:43096->10.133.0.52:15021: read: connection reset by peer" start-of-body= Jul 14 10:32:15.093684 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:15.092472 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:43096->10.133.0.52:15021: read: connection reset by peer" Jul 14 10:32:16.008766 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.008729 2569 generic.go:356] "Generic (PLEG): container finished" podID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerID="66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9" exitCode=0 Jul 14 10:32:16.009039 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.009016 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerDied","Data":"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9"} Jul 14 10:32:16.009654 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.009633 2569 scope.go:117] "RemoveContainer" containerID="66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9" Jul 14 10:32:16.017534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.016735 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:16.017534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.016821 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:16.017534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:16.016867 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:17.020845 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:17.020794 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerStarted","Data":"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d"} Jul 14 10:32:17.899537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:17.899497 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:32:17.899753 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:17.899609 2569 patch_prober.go:28] interesting pod/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:32:17.899753 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:17.899653 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 10:32:18.899302 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:18.899264 2569 patch_prober.go:28] interesting pod/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:32:18.899861 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:18.899830 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 10:32:19.899231 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:19.899191 2569 patch_prober.go:28] interesting pod/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 10:32:19.899444 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:19.899256 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 10:32:20.902552 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:20.902514 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:32:20.903017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:20.902882 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:32:20.903558 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:20.903539 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:32:21.947652 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:21.947620 2569 patch_prober.go:28] interesting pod/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:32:21.948026 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:21.947672 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 10:32:22.053925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.053894 2569 generic.go:356] "Generic (PLEG): container finished" podID="85392f7e-d810-46ea-be30-64657fd89f9e" containerID="7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd" exitCode=0 Jul 14 10:32:22.054079 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.053963 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerDied","Data":"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd"} Jul 14 10:32:22.054488 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.054471 2569 scope.go:117] "RemoveContainer" containerID="7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd" Jul 14 10:32:22.055413 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.055388 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:22.055474 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.055454 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:22.055512 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:22.055491 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:32:23.061805 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:23.061772 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerStarted","Data":"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527"} Jul 14 10:32:23.306067 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:23.306030 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:23.306500 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:23.306471 2569 patch_prober.go:28] interesting pod/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:32:23.306714 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:23.306683 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 10:32:24.305937 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:24.305892 2569 patch_prober.go:28] interesting pod/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:32:24.306408 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:24.305957 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 10:32:25.306281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:25.306244 2569 patch_prober.go:28] interesting pod/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:32:25.306852 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:25.306819 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 10:32:26.305866 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:26.305830 2569 patch_prober.go:28] interesting pod/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 10:32:26.306181 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:26.306130 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 10:32:27.309357 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:27.309323 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:27.309884 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:27.309607 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:32:27.310323 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:32:27.310303 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:33:39.555335 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:39.555301 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:33:39.555814 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:39.555523 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" containerID="cri-o://dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d" gracePeriod=30 Jul 14 10:33:44.812863 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.812837 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:33:44.904434 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904368 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904434 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904403 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904434 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904428 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904448 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904513 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904555 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904580 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904606 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6scb\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904919 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904696 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo\") pod \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\" (UID: \"3d6969c9-b9de-4cf7-91db-4b45a21af85b\") " Jul 14 10:33:44.904919 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904707 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:44.904919 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904892 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:44.904919 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904812 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:33:44.905085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.904995 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:44.905085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.905014 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:44.905085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.905027 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:44.905309 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.905267 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:44.905473 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.905321 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:44.906888 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.906866 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:33:44.906988 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.906905 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:33:44.907132 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.907116 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb" (OuterVolumeSpecName: "kube-api-access-v6scb") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "kube-api-access-v6scb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:33:44.907222 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:44.907178 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d6969c9-b9de-4cf7-91db-4b45a21af85b" (UID: "3d6969c9-b9de-4cf7-91db-4b45a21af85b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:45.005599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005572 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.005599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005597 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.005722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005606 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.005722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005614 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d6969c9-b9de-4cf7-91db-4b45a21af85b-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.005722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005623 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6scb\" (UniqueName: \"kubernetes.io/projected/3d6969c9-b9de-4cf7-91db-4b45a21af85b-kube-api-access-v6scb\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.005722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.005632 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d6969c9-b9de-4cf7-91db-4b45a21af85b-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:45.515350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.515318 2569 generic.go:356] "Generic (PLEG): container finished" podID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerID="dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d" exitCode=0 Jul 14 10:33:45.515515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.515369 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" Jul 14 10:33:45.515515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.515410 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerDied","Data":"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d"} Jul 14 10:33:45.515515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.515468 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv" event={"ID":"3d6969c9-b9de-4cf7-91db-4b45a21af85b","Type":"ContainerDied","Data":"7f73f10d9ec95a08e7fde4cd3e07097fbed85dea49597e1fe869cef8087e2355"} Jul 14 10:33:45.515515 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.515493 2569 scope.go:117] "RemoveContainer" containerID="dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d" Jul 14 10:33:45.527407 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.525694 2569 scope.go:117] "RemoveContainer" containerID="66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9" Jul 14 10:33:45.534880 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.534863 2569 scope.go:117] "RemoveContainer" containerID="dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d" Jul 14 10:33:45.535143 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:33:45.535126 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d\": container with ID starting with dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d not found: ID does not exist" containerID="dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d" Jul 14 10:33:45.535200 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.535167 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d"} err="failed to get container status \"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d\": rpc error: code = NotFound desc = could not find container \"dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d\": container with ID starting with dd362374e62d0112a886242450ec1a810ce3674eb8fdc71d9bc7092a6082f32d not found: ID does not exist" Jul 14 10:33:45.535200 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.535184 2569 scope.go:117] "RemoveContainer" containerID="66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9" Jul 14 10:33:45.535394 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:33:45.535376 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9\": container with ID starting with 66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9 not found: ID does not exist" containerID="66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9" Jul 14 10:33:45.535430 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.535398 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9"} err="failed to get container status \"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9\": rpc error: code = NotFound desc = could not find container \"66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9\": container with ID starting with 66a075006ad21cb1bc5ad03c339964f792e13ac609aa3ca095752cb03d64bfd9 not found: ID does not exist" Jul 14 10:33:45.539332 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.539312 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:33:45.542663 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.542641 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-37ad9d-openshift-default-565787d6fc-4fknv"] Jul 14 10:33:45.872078 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:45.871992 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" path="/var/lib/kubelet/pods/3d6969c9-b9de-4cf7-91db-4b45a21af85b/volumes" Jul 14 10:33:47.140432 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.140400 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:33:47.140829 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.140599 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" containerID="cri-o://2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527" gracePeriod=30 Jul 14 10:33:47.978206 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.978175 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:33:47.978554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.978541 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:47.978600 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.978556 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:47.978635 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.978628 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:47.978668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.978636 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:47.982330 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.982312 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:47.985619 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.985600 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2fed84-openshift-default-dockercfg-wtjzz\"" Jul 14 10:33:47.996544 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:47.996522 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:33:48.132570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132571 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132591 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132609 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjv6\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132789 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132862 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132817 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.132960 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.132865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233407 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233329 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233407 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233374 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233428 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233448 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233468 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233521 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233594 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjv6\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.233887 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233768 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.234241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233874 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.234241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.233919 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.234241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.234118 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.234241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.234135 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.235833 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.235810 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.235925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.235910 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.242968 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.242946 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.243200 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.243182 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjv6\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6\") pod \"e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.293248 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.293221 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:48.432797 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.432773 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:33:48.434653 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:33:48.434621 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb749eeee_a2b6_46d6_b801_eb75261836d5.slice/crio-0c79175c287999130dca46c823d7c891a701d6994394969c0f541b981bde0502 WatchSource:0}: Error finding container 0c79175c287999130dca46c823d7c891a701d6994394969c0f541b981bde0502: Status 404 returned error can't find the container with id 0c79175c287999130dca46c823d7c891a701d6994394969c0f541b981bde0502 Jul 14 10:33:48.436712 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.436683 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:48.436778 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.436749 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:48.436818 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.436780 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:48.537101 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.537055 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerStarted","Data":"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d"} Jul 14 10:33:48.537294 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.537116 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerStarted","Data":"0c79175c287999130dca46c823d7c891a701d6994394969c0f541b981bde0502"} Jul 14 10:33:48.581805 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:48.581755 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podStartSLOduration=1.581733442 podStartE2EDuration="1.581733442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:33:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:33:48.58070263 +0000 UTC m=+2013.214766506" watchObservedRunningTime="2026-07-14 10:33:48.581733442 +0000 UTC m=+2013.215797283" Jul 14 10:33:49.293511 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:49.293478 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:49.298880 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:49.298856 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:49.542832 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:49.542803 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:49.543628 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:49.543577 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:33:52.391890 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.391866 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:33:52.562041 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.562009 2569 generic.go:356] "Generic (PLEG): container finished" podID="85392f7e-d810-46ea-be30-64657fd89f9e" containerID="2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527" exitCode=0 Jul 14 10:33:52.562210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.562055 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" Jul 14 10:33:52.562210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.562055 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerDied","Data":"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527"} Jul 14 10:33:52.562210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.562117 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r" event={"ID":"85392f7e-d810-46ea-be30-64657fd89f9e","Type":"ContainerDied","Data":"018f892672d668ce2eef055d12826ba96625b85d9268fc7c0545a657eb5ff86c"} Jul 14 10:33:52.562210 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.562173 2569 scope.go:117] "RemoveContainer" containerID="2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527" Jul 14 10:33:52.571301 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.571284 2569 scope.go:117] "RemoveContainer" containerID="7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd" Jul 14 10:33:52.574388 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574369 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574492 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574418 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574492 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574463 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574518 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574558 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574601 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574601 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:52.574701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574634 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574666 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574896 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574719 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqrhg\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg\") pod \"85392f7e-d810-46ea-be30-64657fd89f9e\" (UID: \"85392f7e-d810-46ea-be30-64657fd89f9e\") " Jul 14 10:33:52.574951 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574900 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:52.575010 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.574994 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:33:52.575095 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.575075 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.575176 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.575099 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.575176 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.575113 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85392f7e-d810-46ea-be30-64657fd89f9e-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.575320 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.575301 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:52.575613 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.575577 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:52.577046 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.577023 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg" (OuterVolumeSpecName: "kube-api-access-vqrhg") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "kube-api-access-vqrhg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:33:52.577175 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.577074 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:33:52.577175 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.577139 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:33:52.577282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.577184 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "85392f7e-d810-46ea-be30-64657fd89f9e" (UID: "85392f7e-d810-46ea-be30-64657fd89f9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:33:52.579943 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.579925 2569 scope.go:117] "RemoveContainer" containerID="2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527" Jul 14 10:33:52.580213 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:33:52.580189 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527\": container with ID starting with 2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527 not found: ID does not exist" containerID="2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527" Jul 14 10:33:52.580307 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.580218 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527"} err="failed to get container status \"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527\": rpc error: code = NotFound desc = could not find container \"2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527\": container with ID starting with 2891b4b7ae7cb9d06c194cc1ce10b231a97219de4e77dd52acad5e1b4a7d1527 not found: ID does not exist" Jul 14 10:33:52.580307 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.580234 2569 scope.go:117] "RemoveContainer" containerID="7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd" Jul 14 10:33:52.580503 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:33:52.580477 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd\": container with ID starting with 7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd not found: ID does not exist" containerID="7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd" Jul 14 10:33:52.580549 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.580513 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd"} err="failed to get container status \"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd\": rpc error: code = NotFound desc = could not find container \"7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd\": container with ID starting with 7980b975a7d4f2ab6a783af42f9b094352be3bf3b1c0066078c7e5fcc78690dd not found: ID does not exist" Jul 14 10:33:52.675915 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675895 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.675915 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675916 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.676040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675925 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.676040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675934 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85392f7e-d810-46ea-be30-64657fd89f9e-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.676040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675942 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqrhg\" (UniqueName: \"kubernetes.io/projected/85392f7e-d810-46ea-be30-64657fd89f9e-kube-api-access-vqrhg\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.676040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.675949 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85392f7e-d810-46ea-be30-64657fd89f9e-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:33:52.886792 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.886770 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:33:52.891276 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:52.891252 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1242c6-openshift-default-f485fcc6c-98n8r"] Jul 14 10:33:53.871100 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:53.871064 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" path="/var/lib/kubelet/pods/85392f7e-d810-46ea-be30-64657fd89f9e/volumes" Jul 14 10:33:55.690226 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690190 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:33:55.690806 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690787 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:55.690870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690810 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6969c9-b9de-4cf7-91db-4b45a21af85b" containerName="istio-proxy" Jul 14 10:33:55.690870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690823 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:33:55.690870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690830 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:33:55.690870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690848 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:33:55.690870 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690855 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:33:55.691035 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.690950 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:33:55.694017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.693997 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.696818 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.696795 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-56044a-openshift-default-dockercfg-zwffn\"" Jul 14 10:33:55.704581 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704545 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704598 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704671 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs2fc\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704845 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704874 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.704925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704899 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.705136 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.704977 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.713898 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.710065 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:33:55.806571 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.806525 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807077 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807052 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807303 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807412 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807383 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807595 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807424 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807635 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807642 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807676 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs2fc\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.806953 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807811 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807843 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.807883 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.807862 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.808320 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.808292 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.810118 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.810092 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.810537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.810516 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.816357 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.816336 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:55.816608 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:55.816593 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs2fc\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc\") pod \"e2e-api-b-56044a-openshift-default-754c4c777-vfl2g\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:56.004715 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.004646 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:56.145325 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.145296 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:33:56.146609 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:33:56.146581 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d42aa6_7fe8_4e66_8836_f31c0408668b.slice/crio-7866edc5070eb6a7dd52ca7c4d44e368602c428d5f159eac9d8481199c39cab3 WatchSource:0}: Error finding container 7866edc5070eb6a7dd52ca7c4d44e368602c428d5f159eac9d8481199c39cab3: Status 404 returned error can't find the container with id 7866edc5070eb6a7dd52ca7c4d44e368602c428d5f159eac9d8481199c39cab3 Jul 14 10:33:56.148818 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.148785 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:56.148900 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.148845 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:56.148900 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.148873 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:33:56.590439 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.590397 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerStarted","Data":"eae68736429cd637ab200487444a6a0d94b765048df708b01b4b8d589be9ec44"} Jul 14 10:33:56.590636 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.590462 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerStarted","Data":"7866edc5070eb6a7dd52ca7c4d44e368602c428d5f159eac9d8481199c39cab3"} Jul 14 10:33:56.617115 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:56.617067 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podStartSLOduration=1.617052554 podStartE2EDuration="1.617052554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:33:56.614336316 +0000 UTC m=+2021.248400158" watchObservedRunningTime="2026-07-14 10:33:56.617052554 +0000 UTC m=+2021.251116394" Jul 14 10:33:57.005277 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:57.005245 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:57.010095 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:57.010072 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:57.595945 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:57.595917 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:33:57.596819 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:33:57.596804 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:05.584314 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:05.584272 2569 patch_prober.go:28] interesting pod/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 10:34:05.584719 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:05.584344 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 10:34:06.654534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.654500 2569 generic.go:356] "Generic (PLEG): container finished" podID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerID="12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d" exitCode=0 Jul 14 10:34:06.655024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.654577 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerDied","Data":"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d"} Jul 14 10:34:06.655024 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.654994 2569 scope.go:117] "RemoveContainer" containerID="12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d" Jul 14 10:34:06.656062 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.656034 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:06.656233 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.656098 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:06.656233 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:06.656131 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:07.661773 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:07.661741 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerStarted","Data":"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6"} Jul 14 10:34:08.294385 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:08.294348 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:08.294604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:08.294540 2569 patch_prober.go:28] interesting pod/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 10:34:08.294604 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:08.294586 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 10:34:09.293713 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:09.293682 2569 patch_prober.go:28] interesting pod/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 10:34:09.294277 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:09.294249 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 10:34:10.294237 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:10.294192 2569 patch_prober.go:28] interesting pod/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 10:34:10.294641 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:10.294251 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 10:34:11.293754 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:11.293717 2569 patch_prober.go:28] interesting pod/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 10:34:11.293959 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:11.293771 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 10:34:12.297172 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:12.297122 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:12.297563 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:12.297543 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:12.298125 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:12.298104 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:13.371560 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.371483 2569 patch_prober.go:28] interesting pod/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:44988->10.133.0.55:15021: read: connection reset by peer" start-of-body= Jul 14 10:34:13.371560 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.371543 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": read tcp 10.133.0.2:44988->10.133.0.55:15021: read: connection reset by peer" Jul 14 10:34:13.696437 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.696410 2569 generic.go:356] "Generic (PLEG): container finished" podID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerID="eae68736429cd637ab200487444a6a0d94b765048df708b01b4b8d589be9ec44" exitCode=0 Jul 14 10:34:13.696626 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.696491 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerDied","Data":"eae68736429cd637ab200487444a6a0d94b765048df708b01b4b8d589be9ec44"} Jul 14 10:34:13.696999 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.696981 2569 scope.go:117] "RemoveContainer" containerID="eae68736429cd637ab200487444a6a0d94b765048df708b01b4b8d589be9ec44" Jul 14 10:34:13.697926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.697897 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:13.698047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.697972 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:13.698047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:13.698016 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:14.712135 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:14.712099 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerStarted","Data":"9e050fee0ef5cf6fd200af4110d28741af1e6e23e3e9506a1fcadbc0fd3b0d1c"} Jul 14 10:34:15.005034 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:15.004938 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:15.005286 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:15.005255 2569 patch_prober.go:28] interesting pod/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 10:34:15.005489 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:15.005462 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 10:34:16.005392 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:16.005355 2569 patch_prober.go:28] interesting pod/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 10:34:16.005831 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:16.005419 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 10:34:17.007564 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:17.007520 2569 patch_prober.go:28] interesting pod/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 10:34:17.008032 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:17.007584 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 10:34:18.005065 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:18.005028 2569 patch_prober.go:28] interesting pod/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 10:34:18.005373 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:18.005333 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 10:34:18.236243 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:18.236206 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:34:18.236623 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:18.236446 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" containerID="cri-o://3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6" gracePeriod=30 Jul 14 10:34:19.008695 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:19.008664 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:19.009051 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:19.009029 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:19.009768 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:19.009749 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:23.489380 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.489354 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:23.493094 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493072 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493216 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493115 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493216 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493193 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493229 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493261 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493328 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493304 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493335 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:23.493480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493351 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tjv6\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493385 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493480 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493453 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:23.493696 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493507 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy\") pod \"b749eeee-a2b6-46d6-b801-eb75261836d5\" (UID: \"b749eeee-a2b6-46d6-b801-eb75261836d5\") " Jul 14 10:34:23.493696 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493618 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:34:23.493817 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493755 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:23.493950 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493929 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.494020 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493955 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.494020 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493969 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b749eeee-a2b6-46d6-b801-eb75261836d5-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.494020 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493968 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:23.494020 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.493983 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.495540 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.495485 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:34:23.495638 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.495566 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:23.495697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.495638 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6" (OuterVolumeSpecName: "kube-api-access-4tjv6") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "kube-api-access-4tjv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:34:23.495697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.495641 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b749eeee-a2b6-46d6-b801-eb75261836d5" (UID: "b749eeee-a2b6-46d6-b801-eb75261836d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:34:23.595244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.595220 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.595244 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.595243 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.595390 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.595251 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tjv6\" (UniqueName: \"kubernetes.io/projected/b749eeee-a2b6-46d6-b801-eb75261836d5-kube-api-access-4tjv6\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.595390 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.595261 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.595390 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.595269 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b749eeee-a2b6-46d6-b801-eb75261836d5-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:23.766854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.766777 2569 generic.go:356] "Generic (PLEG): container finished" podID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerID="3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6" exitCode=0 Jul 14 10:34:23.766854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.766822 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerDied","Data":"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6"} Jul 14 10:34:23.766854 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.766823 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" Jul 14 10:34:23.767069 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.766864 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc" event={"ID":"b749eeee-a2b6-46d6-b801-eb75261836d5","Type":"ContainerDied","Data":"0c79175c287999130dca46c823d7c891a701d6994394969c0f541b981bde0502"} Jul 14 10:34:23.767069 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.766889 2569 scope.go:117] "RemoveContainer" containerID="3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6" Jul 14 10:34:23.776192 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.776177 2569 scope.go:117] "RemoveContainer" containerID="12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d" Jul 14 10:34:23.783906 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.783888 2569 scope.go:117] "RemoveContainer" containerID="3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6" Jul 14 10:34:23.784161 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:34:23.784131 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6\": container with ID starting with 3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6 not found: ID does not exist" containerID="3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6" Jul 14 10:34:23.784218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.784171 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6"} err="failed to get container status \"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6\": rpc error: code = NotFound desc = could not find container \"3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6\": container with ID starting with 3cb4c0555b26ee7f291fa82ddda692938b79d08ad75875ac0d447db1665240d6 not found: ID does not exist" Jul 14 10:34:23.784218 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.784192 2569 scope.go:117] "RemoveContainer" containerID="12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d" Jul 14 10:34:23.784445 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:34:23.784428 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d\": container with ID starting with 12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d not found: ID does not exist" containerID="12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d" Jul 14 10:34:23.784488 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.784451 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d"} err="failed to get container status \"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d\": rpc error: code = NotFound desc = could not find container \"12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d\": container with ID starting with 12e649dcc8513b13e863aebc8ddc466f46cffa81879e5d28c470eedcd8f5218d not found: ID does not exist" Jul 14 10:34:23.792534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.792494 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:34:23.812591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.812565 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2fed84-openshift-default-c9ffddb57-2dqvc"] Jul 14 10:34:23.871501 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:23.871478 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" path="/var/lib/kubelet/pods/b749eeee-a2b6-46d6-b801-eb75261836d5/volumes" Jul 14 10:34:25.635223 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:25.635190 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:34:25.635599 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:25.635410 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" containerID="cri-o://9e050fee0ef5cf6fd200af4110d28741af1e6e23e3e9506a1fcadbc0fd3b0d1c" gracePeriod=30 Jul 14 10:34:26.463555 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.463514 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:34:26.463959 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.463943 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:26.464025 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.463962 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:26.464067 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.464040 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:26.464067 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.464047 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:26.464067 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.464058 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="85392f7e-d810-46ea-be30-64657fd89f9e" containerName="istio-proxy" Jul 14 10:34:26.467256 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.467236 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.469878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.469856 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-817aae-openshift-default-dockercfg-j7k56\"" Jul 14 10:34:26.480426 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.480404 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:34:26.536168 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536129 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536285 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536201 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536285 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536259 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536308 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536336 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536368 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536464 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536421 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.536627 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.536472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfqv\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.637966 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.637934 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.637972 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.637996 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638017 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638035 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638062 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638094 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfqv\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638221 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638388 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638455 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638930 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638665 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638930 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638693 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.638930 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.638885 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.640245 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.640228 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.640660 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.640641 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.647601 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.647579 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfqv\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.647907 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.647886 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token\") pod \"e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.777820 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.777758 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:26.918255 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.918222 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:34:26.919480 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:34:26.919445 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c971fce_8f6c_476e_a5ce_b562bdbd92be.slice/crio-7b9f0d65ada81d0ac85e26ed5d2355906521ee5c55dc9cbeabdcd06424e8bf28 WatchSource:0}: Error finding container 7b9f0d65ada81d0ac85e26ed5d2355906521ee5c55dc9cbeabdcd06424e8bf28: Status 404 returned error can't find the container with id 7b9f0d65ada81d0ac85e26ed5d2355906521ee5c55dc9cbeabdcd06424e8bf28 Jul 14 10:34:26.921625 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.921592 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:26.921723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.921657 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:26.921723 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:26.921689 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:27.800788 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:27.800756 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerStarted","Data":"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c"} Jul 14 10:34:27.801191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:27.800804 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerStarted","Data":"7b9f0d65ada81d0ac85e26ed5d2355906521ee5c55dc9cbeabdcd06424e8bf28"} Jul 14 10:34:27.827698 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:27.827652 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" podStartSLOduration=1.827638737 podStartE2EDuration="1.827638737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:34:27.825883497 +0000 UTC m=+2052.459947340" watchObservedRunningTime="2026-07-14 10:34:27.827638737 +0000 UTC m=+2052.461702577" Jul 14 10:34:28.778253 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:28.778218 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:28.782954 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:28.782931 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:28.806425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:28.806393 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:28.807082 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:28.807065 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:30.821901 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.821866 2569 generic.go:356] "Generic (PLEG): container finished" podID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerID="9e050fee0ef5cf6fd200af4110d28741af1e6e23e3e9506a1fcadbc0fd3b0d1c" exitCode=0 Jul 14 10:34:30.822291 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.821941 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerDied","Data":"9e050fee0ef5cf6fd200af4110d28741af1e6e23e3e9506a1fcadbc0fd3b0d1c"} Jul 14 10:34:30.822291 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.822020 2569 scope.go:117] "RemoveContainer" containerID="eae68736429cd637ab200487444a6a0d94b765048df708b01b4b8d589be9ec44" Jul 14 10:34:30.889130 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.889107 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:30.981713 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981684 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.981882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981723 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.981882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981750 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs2fc\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.981882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981773 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.981882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981829 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.981882 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981871 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.982139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981901 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.982139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.981980 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.982139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982010 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:30.982139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982017 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy\") pod \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\" (UID: \"d6d42aa6-7fe8-4e66-8836-f31c0408668b\") " Jul 14 10:34:30.982139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982125 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:30.982350 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982312 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:34:30.982469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982411 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:30.982469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982433 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:30.982469 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982448 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:30.982710 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982467 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:30.982770 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.982737 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:30.984099 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.984079 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:34:30.984259 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.984164 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc" (OuterVolumeSpecName: "kube-api-access-zs2fc") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "kube-api-access-zs2fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:34:30.984527 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.984508 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:34:30.984567 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:30.984511 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6d42aa6-7fe8-4e66-8836-f31c0408668b" (UID: "d6d42aa6-7fe8-4e66-8836-f31c0408668b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:34:31.083336 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083310 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.083336 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083334 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs2fc\" (UniqueName: \"kubernetes.io/projected/d6d42aa6-7fe8-4e66-8836-f31c0408668b-kube-api-access-zs2fc\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.083473 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083343 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.083473 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083352 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.083473 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083360 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.083473 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.083368 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6d42aa6-7fe8-4e66-8836-f31c0408668b-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:34:31.831808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.831774 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" event={"ID":"d6d42aa6-7fe8-4e66-8836-f31c0408668b","Type":"ContainerDied","Data":"7866edc5070eb6a7dd52ca7c4d44e368602c428d5f159eac9d8481199c39cab3"} Jul 14 10:34:31.831808 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.831797 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g" Jul 14 10:34:31.832321 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.831828 2569 scope.go:117] "RemoveContainer" containerID="9e050fee0ef5cf6fd200af4110d28741af1e6e23e3e9506a1fcadbc0fd3b0d1c" Jul 14 10:34:31.854798 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.854771 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:34:31.859807 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.859782 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-56044a-openshift-default-754c4c777-vfl2g"] Jul 14 10:34:31.870234 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:31.870214 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" path="/var/lib/kubelet/pods/d6d42aa6-7fe8-4e66-8836-f31c0408668b/volumes" Jul 14 10:34:34.736752 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.736712 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:34:34.737239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737216 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:34.737239 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737234 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b749eeee-a2b6-46d6-b801-eb75261836d5" containerName="istio-proxy" Jul 14 10:34:34.737324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737249 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.737324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737257 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.737324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737281 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.737324 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737291 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.737448 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737402 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.737448 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.737415 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6d42aa6-7fe8-4e66-8836-f31c0408668b" containerName="istio-proxy" Jul 14 10:34:34.740668 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.740649 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.743230 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.743205 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-684339-openshift-default-dockercfg-nxpgp\"" Jul 14 10:34:34.750673 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.750650 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:34:34.813495 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813520 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8646\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813596 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813647 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813621 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813827 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813650 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813827 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813718 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813827 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.813827 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.813758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914693 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914731 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914734 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914759 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914797 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914852 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914899 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8646\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.914979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.914927 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915017 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915058 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915180 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915471 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915322 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915471 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915417 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915620 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915597 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.915740 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.915725 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.917122 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.917089 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.917418 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.917400 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.923615 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.923592 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:34.923737 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:34.923715 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8646\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646\") pod \"e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:35.052214 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.052090 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:35.196181 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.196137 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:34:35.197689 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:34:35.197664 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b70273_7ff7_4e00_98aa_6d0b5c2316c0.slice/crio-2e48f12b0d1d6051c335eb2227bb8500adfec6f3147374ae386b51aeb2246fee WatchSource:0}: Error finding container 2e48f12b0d1d6051c335eb2227bb8500adfec6f3147374ae386b51aeb2246fee: Status 404 returned error can't find the container with id 2e48f12b0d1d6051c335eb2227bb8500adfec6f3147374ae386b51aeb2246fee Jul 14 10:34:35.200191 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.200141 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:35.200268 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.200224 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:35.200268 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.200251 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:35.858017 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.857982 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerStarted","Data":"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb"} Jul 14 10:34:35.858531 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.858041 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerStarted","Data":"2e48f12b0d1d6051c335eb2227bb8500adfec6f3147374ae386b51aeb2246fee"} Jul 14 10:34:35.883038 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:35.882990 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podStartSLOduration=1.882976705 podStartE2EDuration="1.882976705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 10:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:34:35.880706451 +0000 UTC m=+2060.514770303" watchObservedRunningTime="2026-07-14 10:34:35.882976705 +0000 UTC m=+2060.517040546" Jul 14 10:34:36.053233 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:36.053198 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:36.058094 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:36.058069 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:36.863377 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:36.863340 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:36.864241 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:36.864225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:42.896782 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.896744 2569 generic.go:356] "Generic (PLEG): container finished" podID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerID="23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c" exitCode=0 Jul 14 10:34:42.897250 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.896788 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerDied","Data":"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c"} Jul 14 10:34:42.897250 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.897193 2569 scope.go:117] "RemoveContainer" containerID="23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c" Jul 14 10:34:42.898223 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.898194 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:42.898293 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.898275 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:42.898344 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:42.898328 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:43.807465 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:43.807423 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:43.909222 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:43.909186 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerStarted","Data":"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43"} Jul 14 10:34:44.777982 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:44.777923 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:44.779212 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:44.779184 2569 patch_prober.go:28] interesting pod/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 10:34:44.779320 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:44.779233 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 10:34:45.778271 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:45.778229 2569 patch_prober.go:28] interesting pod/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 10:34:45.778645 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:45.778289 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 10:34:46.778279 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:46.778241 2569 patch_prober.go:28] interesting pod/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 10:34:46.778666 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:46.778309 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 10:34:47.781632 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:47.781597 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:47.782043 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:47.781966 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:47.782545 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:47.782520 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:34:51.865265 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.865225 2569 patch_prober.go:28] interesting pod/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 10:34:51.865664 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.865295 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 10:34:51.968061 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.968033 2569 generic.go:356] "Generic (PLEG): container finished" podID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerID="84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb" exitCode=0 Jul 14 10:34:51.968209 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.968104 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerDied","Data":"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb"} Jul 14 10:34:51.968453 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.968439 2569 scope.go:117] "RemoveContainer" containerID="84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb" Jul 14 10:34:51.969356 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.969329 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:51.969443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.969390 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:51.969443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:51.969421 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 10:34:52.975413 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:52.975378 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerStarted","Data":"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f"} Jul 14 10:34:53.053034 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:53.052995 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:53.053248 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:53.053225 2569 patch_prober.go:28] interesting pod/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 10:34:53.053314 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:53.053277 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 10:34:54.052428 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:54.052393 2569 patch_prober.go:28] interesting pod/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 10:34:54.052803 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:54.052462 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 10:34:55.053425 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:55.053392 2569 patch_prober.go:28] interesting pod/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 10:34:55.053886 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:55.053452 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 10:34:56.052519 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:56.052485 2569 patch_prober.go:28] interesting pod/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 10:34:56.052690 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:56.052538 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 10:34:57.056421 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:57.056391 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:57.056867 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:57.056669 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:34:57.057338 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:34:57.057322 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:35:16.015680 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:35:16.015651 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:35:16.028116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:35:16.028090 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:40:16.055096 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:40:16.055062 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:40:16.068547 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:40:16.068523 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5zh8_784d21c1-2b89-4930-acae-af220fb6fe25/ovn-acl-logging/0.log" Jul 14 10:41:21.544619 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:21.544585 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:41:21.545139 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:21.544787 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" containerID="cri-o://c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43" gracePeriod=30 Jul 14 10:41:26.801019 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.800995 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:41:26.904905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.904840 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.904905 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.904878 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.904906 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905009 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905042 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905103 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905082 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905348 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905112 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjfqv\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905348 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905179 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905348 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905214 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket\") pod \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\" (UID: \"8c971fce-8f6c-476e-a5ce-b562bdbd92be\") " Jul 14 10:41:26.905510 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905321 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:26.905580 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905515 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:26.905639 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905566 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:41:26.905639 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905602 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:26.905639 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905622 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:26.905746 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.905701 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:26.906085 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.906068 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:26.907558 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.907529 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:41:26.907671 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.907577 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:26.907671 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.907604 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:41:26.907671 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:26.907628 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv" (OuterVolumeSpecName: "kube-api-access-pjfqv") pod "8c971fce-8f6c-476e-a5ce-b562bdbd92be" (UID: "8c971fce-8f6c-476e-a5ce-b562bdbd92be"). InnerVolumeSpecName "kube-api-access-pjfqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:41:27.006746 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006721 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006746 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006744 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006879 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006753 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006879 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006760 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006879 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006768 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjfqv\" (UniqueName: \"kubernetes.io/projected/8c971fce-8f6c-476e-a5ce-b562bdbd92be-kube-api-access-pjfqv\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006879 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006777 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.006879 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.006785 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c971fce-8f6c-476e-a5ce-b562bdbd92be-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:27.169516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.169449 2569 generic.go:356] "Generic (PLEG): container finished" podID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerID="c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43" exitCode=0 Jul 14 10:41:27.169516 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.169499 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" Jul 14 10:41:27.169697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.169530 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerDied","Data":"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43"} Jul 14 10:41:27.169697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.169590 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l" event={"ID":"8c971fce-8f6c-476e-a5ce-b562bdbd92be","Type":"ContainerDied","Data":"7b9f0d65ada81d0ac85e26ed5d2355906521ee5c55dc9cbeabdcd06424e8bf28"} Jul 14 10:41:27.169697 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.169602 2569 scope.go:117] "RemoveContainer" containerID="c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43" Jul 14 10:41:27.179109 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.179093 2569 scope.go:117] "RemoveContainer" containerID="23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c" Jul 14 10:41:27.187541 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.187506 2569 scope.go:117] "RemoveContainer" containerID="c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43" Jul 14 10:41:27.187767 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:41:27.187751 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43\": container with ID starting with c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43 not found: ID does not exist" containerID="c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43" Jul 14 10:41:27.187813 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.187777 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43"} err="failed to get container status \"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43\": rpc error: code = NotFound desc = could not find container \"c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43\": container with ID starting with c911b7b83aaff7e12e04857a60896937ed47008c15a18a5308b864fec7f73b43 not found: ID does not exist" Jul 14 10:41:27.187813 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.187794 2569 scope.go:117] "RemoveContainer" containerID="23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c" Jul 14 10:41:27.188016 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:41:27.187999 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c\": container with ID starting with 23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c not found: ID does not exist" containerID="23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c" Jul 14 10:41:27.188076 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.188027 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c"} err="failed to get container status \"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c\": rpc error: code = NotFound desc = could not find container \"23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c\": container with ID starting with 23b2262c9bf0df26dd6087f514ac3f7d1325d482cf1ec931dae082306a940c1c not found: ID does not exist" Jul 14 10:41:27.191884 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.191863 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:41:27.195254 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.195233 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-817aae-openshift-default-854c7d488c-sx79l"] Jul 14 10:41:27.870982 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:27.870948 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" path="/var/lib/kubelet/pods/8c971fce-8f6c-476e-a5ce-b562bdbd92be/volumes" Jul 14 10:41:29.164093 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:29.164060 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:41:29.164570 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:29.164292 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" containerID="cri-o://c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f" gracePeriod=30 Jul 14 10:41:34.412714 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.412687 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:41:34.578743 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578718 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.578871 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578759 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.578871 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578788 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.578871 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578838 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579002 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578879 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579002 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578919 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8646\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579002 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578942 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579002 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.578998 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579236 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579001 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:34.579236 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579024 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo\") pod \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\" (UID: \"86b70273-7ff7-4e00-98aa-6d0b5c2316c0\") " Jul 14 10:41:34.579236 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579139 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:34.579236 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579185 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 10:41:34.579440 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579376 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:34.579517 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579488 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-credential-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.579517 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579505 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-socket\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.579617 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579519 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istiod-ca-cert\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.579617 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579532 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-workload-certs\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.579739 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.579695 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:34.581116 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.581089 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 10:41:34.581250 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.581143 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:41:34.581497 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.581476 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 10:41:34.581543 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.581492 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646" (OuterVolumeSpecName: "kube-api-access-f8646") pod "86b70273-7ff7-4e00-98aa-6d0b5c2316c0" (UID: "86b70273-7ff7-4e00-98aa-6d0b5c2316c0"). InnerVolumeSpecName "kube-api-access-f8646". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 10:41:34.680740 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.680716 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-podinfo\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.680740 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.680739 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-data\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.680881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.680747 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-token\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.680881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.680755 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8646\" (UniqueName: \"kubernetes.io/projected/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-kube-api-access-f8646\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.680881 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.680763 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86b70273-7ff7-4e00-98aa-6d0b5c2316c0-istio-envoy\") on node \"ip-10-0-141-78.ec2.internal\" DevicePath \"\"" Jul 14 10:41:34.855734 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.855681 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29733735-k6mfs_84f9229e-545c-48be-a7e9-ed5620b55bf7/cleanup/0.log" Jul 14 10:41:34.966169 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:34.966130 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29733750-s5wr8_f1de1108-9f98-42fb-8ec1-07a9eabe2ace/cleanup/0.log" Jul 14 10:41:35.086106 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.086086 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-667d7fccf6-k9rsj_b567d770-0548-448a-bdc1-4b03840a7869/manager/0.log" Jul 14 10:41:35.215939 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.215908 2569 generic.go:356] "Generic (PLEG): container finished" podID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerID="c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f" exitCode=0 Jul 14 10:41:35.216063 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.215958 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" Jul 14 10:41:35.216063 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.215993 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerDied","Data":"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f"} Jul 14 10:41:35.216063 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.216046 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8" event={"ID":"86b70273-7ff7-4e00-98aa-6d0b5c2316c0","Type":"ContainerDied","Data":"2e48f12b0d1d6051c335eb2227bb8500adfec6f3147374ae386b51aeb2246fee"} Jul 14 10:41:35.216201 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.216068 2569 scope.go:117] "RemoveContainer" containerID="c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f" Jul 14 10:41:35.225688 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.225669 2569 scope.go:117] "RemoveContainer" containerID="84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb" Jul 14 10:41:35.236809 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.236750 2569 scope.go:117] "RemoveContainer" containerID="c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f" Jul 14 10:41:35.237281 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:41:35.237065 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f\": container with ID starting with c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f not found: ID does not exist" containerID="c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f" Jul 14 10:41:35.237281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.237106 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f"} err="failed to get container status \"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f\": rpc error: code = NotFound desc = could not find container \"c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f\": container with ID starting with c56e9c95be295215179ede12843cecc36ed1ec66370875cb3a918d458d4ddf5f not found: ID does not exist" Jul 14 10:41:35.237281 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.237129 2569 scope.go:117] "RemoveContainer" containerID="84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb" Jul 14 10:41:35.237551 ip-10-0-141-78 kubenswrapper[2569]: E0714 10:41:35.237446 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb\": container with ID starting with 84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb not found: ID does not exist" containerID="84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb" Jul 14 10:41:35.237551 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.237497 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb"} err="failed to get container status \"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb\": rpc error: code = NotFound desc = could not find container \"84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb\": container with ID starting with 84741c1c08772ca494beae5063fcba971918fa1221bead2dc0e51d38520015bb not found: ID does not exist" Jul 14 10:41:35.240124 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.240105 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:41:35.243836 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.243816 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-684339-openshift-default-5fd5db94c9-pl2b8"] Jul 14 10:41:35.580322 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.580255 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj_e69b7b2c-d086-489a-b005-fa5ba1fdfff6/manager/0.log" Jul 14 10:41:35.586211 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.586187 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj_e69b7b2c-d086-489a-b005-fa5ba1fdfff6/copy-manifests/0.log" Jul 14 10:41:35.870685 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:35.870611 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" path="/var/lib/kubelet/pods/86b70273-7ff7-4e00-98aa-6d0b5c2316c0/volumes" Jul 14 10:41:36.606131 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.606102 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/util/0.log" Jul 14 10:41:36.612466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.612447 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/pull/0.log" Jul 14 10:41:36.618267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.618247 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/extract/0.log" Jul 14 10:41:36.725932 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.725909 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/util/0.log" Jul 14 10:41:36.733115 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.733085 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/pull/0.log" Jul 14 10:41:36.740185 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.740169 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/extract/0.log" Jul 14 10:41:36.851619 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.851598 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/extract/0.log" Jul 14 10:41:36.858209 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.858146 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/util/0.log" Jul 14 10:41:36.864845 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.864817 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/pull/0.log" Jul 14 10:41:36.969979 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.969963 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/util/0.log" Jul 14 10:41:36.976404 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.976387 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/pull/0.log" Jul 14 10:41:36.983040 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:36.983023 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/extract/0.log" Jul 14 10:41:38.358902 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:38.358874 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gzbt4_29b302a7-97d9-44b4-b065-cf6c43bf330f/istio-proxy/0.log" Jul 14 10:41:38.475758 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:38.475727 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-45qvb_6051ad4d-18c2-4c64-80e4-998423db2c9b/discovery/0.log" Jul 14 10:41:38.835722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:38.835688 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/1.log" Jul 14 10:41:39.677242 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:39.677215 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6xl9s_53c913f9-1218-4917-ae27-c2b9e876a501/main/0.log" Jul 14 10:41:39.787622 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:39.787594 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-htcx4_812a40ed-eb22-4af2-895a-a443f134d687/main/0.log" Jul 14 10:41:46.912571 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:46.912536 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7nmln_b68475b9-1f2f-4ad9-875c-ea54c2400cc0/global-pull-secret-syncer/0.log" Jul 14 10:41:47.152806 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:47.152778 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tjjsb_f7773ae2-7435-415e-8bbf-7fa42bdb462e/konnectivity-agent/0.log" Jul 14 10:41:47.289554 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:47.289526 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-78.ec2.internal_5af106930072935ac7f96a0491100845/haproxy/0.log" Jul 14 10:41:51.244726 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.244691 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/extract/0.log" Jul 14 10:41:51.276416 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.276390 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/util/0.log" Jul 14 10:41:51.307398 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.307377 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759244mr_70e6f00a-2c33-40b6-8c80-1f1ba76b2ba8/pull/0.log" Jul 14 10:41:51.335267 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.335245 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/extract/0.log" Jul 14 10:41:51.362277 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.362256 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/util/0.log" Jul 14 10:41:51.384266 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.384245 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0k75qm_cdf87a83-6b48-49fd-9356-437ef3d17848/pull/0.log" Jul 14 10:41:51.426596 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.426580 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/extract/0.log" Jul 14 10:41:51.450002 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.449979 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/util/0.log" Jul 14 10:41:51.474873 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.474853 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pbrv2_ea330f78-f9fb-4c94-8d13-9398fa111f68/pull/0.log" Jul 14 10:41:51.502594 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.502533 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/extract/0.log" Jul 14 10:41:51.529126 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.529110 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/util/0.log" Jul 14 10:41:51.559269 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:51.559251 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12jq9k_e217d45f-032e-461e-b28c-fe195281508b/pull/0.log" Jul 14 10:41:54.181764 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.181721 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/alertmanager/0.log" Jul 14 10:41:54.215273 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.215251 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/config-reloader/0.log" Jul 14 10:41:54.249728 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.249703 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/kube-rbac-proxy-web/0.log" Jul 14 10:41:54.279722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.279704 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/kube-rbac-proxy/0.log" Jul 14 10:41:54.314675 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.314653 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/kube-rbac-proxy-metric/0.log" Jul 14 10:41:54.343081 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.343059 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/prom-label-proxy/0.log" Jul 14 10:41:54.368866 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.368832 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_09340412-bcd5-4780-abb9-2188cd291d3e/init-config-reloader/0.log" Jul 14 10:41:54.575206 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.575181 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55bcfc6b77-56pxm_676119f7-ccf9-42dd-a463-c70636c521d3/metrics-server/0.log" Jul 14 10:41:54.605574 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.605537 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-nm5tq_44a6f46b-05a0-46f8-8015-efc5b5bf31f0/monitoring-plugin/0.log" Jul 14 10:41:54.821597 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.821576 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc45b_45c7857b-1cf4-4805-9f23-a41bf4355042/node-exporter/0.log" Jul 14 10:41:54.846867 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.846848 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc45b_45c7857b-1cf4-4805-9f23-a41bf4355042/kube-rbac-proxy/0.log" Jul 14 10:41:54.872540 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.872524 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rc45b_45c7857b-1cf4-4805-9f23-a41bf4355042/init-textfile/0.log" Jul 14 10:41:54.911375 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.911348 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4mg8z_90eccaaf-0c5e-4e60-8b5b-846e45979f4e/kube-rbac-proxy-main/0.log" Jul 14 10:41:54.934072 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.934053 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4mg8z_90eccaaf-0c5e-4e60-8b5b-846e45979f4e/kube-rbac-proxy-self/0.log" Jul 14 10:41:54.955860 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:54.955844 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4mg8z_90eccaaf-0c5e-4e60-8b5b-846e45979f4e/openshift-state-metrics/0.log" Jul 14 10:41:55.168503 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.168423 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-xdd5t_018133ad-1ded-49af-9302-5f5a9810e78d/prometheus-operator/0.log" Jul 14 10:41:55.194627 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.194602 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-xdd5t_018133ad-1ded-49af-9302-5f5a9810e78d/kube-rbac-proxy/0.log" Jul 14 10:41:55.228200 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.228172 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-hvccf_81f21ca6-03a1-4204-8187-5979c5b36f11/prometheus-operator-admission-webhook/0.log" Jul 14 10:41:55.257672 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.257629 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f846766f-6848q_2fdec8f8-2e75-4b25-88bd-b4762c959af7/telemeter-client/0.log" Jul 14 10:41:55.282722 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.282699 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f846766f-6848q_2fdec8f8-2e75-4b25-88bd-b4762c959af7/reload/0.log" Jul 14 10:41:55.305914 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.305892 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f846766f-6848q_2fdec8f8-2e75-4b25-88bd-b4762c959af7/kube-rbac-proxy/0.log" Jul 14 10:41:55.342047 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.341998 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/thanos-query/0.log" Jul 14 10:41:55.365673 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.365651 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/kube-rbac-proxy-web/0.log" Jul 14 10:41:55.388925 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.388907 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/kube-rbac-proxy/0.log" Jul 14 10:41:55.424304 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.424252 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/prom-label-proxy/0.log" Jul 14 10:41:55.447423 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.447404 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/kube-rbac-proxy-rules/0.log" Jul 14 10:41:55.470949 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:55.470927 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cd48bf654-hztls_3a9d9f49-f235-4942-9ba2-fd59b5396f74/kube-rbac-proxy-metrics/0.log" Jul 14 10:41:56.625537 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.625508 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-vpnjq_d8da39dc-4ed7-4fd9-8221-6bc169ecb783/networking-console-plugin/0.log" Jul 14 10:41:56.916030 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.915937 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8"] Jul 14 10:41:56.916338 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916324 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" Jul 14 10:41:56.916409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916339 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" Jul 14 10:41:56.916409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916366 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" Jul 14 10:41:56.916409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916371 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" Jul 14 10:41:56.916409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916380 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" Jul 14 10:41:56.916409 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916386 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" Jul 14 10:41:56.916597 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916456 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" Jul 14 10:41:56.916597 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916467 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="86b70273-7ff7-4e00-98aa-6d0b5c2316c0" containerName="istio-proxy" Jul 14 10:41:56.916597 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.916478 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c971fce-8f6c-476e-a5ce-b562bdbd92be" containerName="istio-proxy" Jul 14 10:41:56.919978 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.919962 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:56.922701 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.922679 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5rfhj\"/\"openshift-service-ca.crt\"" Jul 14 10:41:56.923776 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.923755 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5rfhj\"/\"default-dockercfg-f5pxx\"" Jul 14 10:41:56.923878 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.923762 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5rfhj\"/\"kube-root-ca.crt\"" Jul 14 10:41:56.932339 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.932316 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8"] Jul 14 10:41:56.960081 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.960059 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-sys\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:56.960188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.960090 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-podres\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:56.960188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.960110 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-proc\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:56.960188 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.960176 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx2cv\" (UniqueName: \"kubernetes.io/projected/c974eadc-31f4-4b29-ba87-7ac53853652d-kube-api-access-nx2cv\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:56.960302 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:56.960254 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-lib-modules\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061443 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061419 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-sys\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-podres\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061471 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-proc\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061503 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx2cv\" (UniqueName: \"kubernetes.io/projected/c974eadc-31f4-4b29-ba87-7ac53853652d-kube-api-access-nx2cv\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061543 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-sys\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-lib-modules\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061591 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061578 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-proc\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061826 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061616 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-podres\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.061826 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.061632 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c974eadc-31f4-4b29-ba87-7ac53853652d-lib-modules\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.069811 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.069787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx2cv\" (UniqueName: \"kubernetes.io/projected/c974eadc-31f4-4b29-ba87-7ac53853652d-kube-api-access-nx2cv\") pod \"perf-node-gather-daemonset-r6kd8\" (UID: \"c974eadc-31f4-4b29-ba87-7ac53853652d\") " pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.230534 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.230502 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:57.366420 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:57.366394 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8"] Jul 14 10:41:57.368088 ip-10-0-141-78 kubenswrapper[2569]: W0714 10:41:57.368047 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc974eadc_31f4_4b29_ba87_7ac53853652d.slice/crio-885ad0d9f9890cd987dce329eb1bd0ee06bc49e9a13c13e817adef82103d0e9a WatchSource:0}: Error finding container 885ad0d9f9890cd987dce329eb1bd0ee06bc49e9a13c13e817adef82103d0e9a: Status 404 returned error can't find the container with id 885ad0d9f9890cd987dce329eb1bd0ee06bc49e9a13c13e817adef82103d0e9a Jul 14 10:41:58.340331 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:58.340293 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" event={"ID":"c974eadc-31f4-4b29-ba87-7ac53853652d","Type":"ContainerStarted","Data":"4c466abb9a4d07a4b6b9bd4a29c7ad6dfcbe88f56ed64e54af8d3b4458f09438"} Jul 14 10:41:58.340742 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:58.340357 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:41:58.340742 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:58.340377 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" event={"ID":"c974eadc-31f4-4b29-ba87-7ac53853652d","Type":"ContainerStarted","Data":"885ad0d9f9890cd987dce329eb1bd0ee06bc49e9a13c13e817adef82103d0e9a"} Jul 14 10:41:58.362520 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:58.362472 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" podStartSLOduration=2.362453008 podStartE2EDuration="2.362453008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 10:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 10:41:58.360031824 +0000 UTC m=+2502.994095664" watchObservedRunningTime="2026-07-14 10:41:58.362453008 +0000 UTC m=+2502.996516849" Jul 14 10:41:59.734658 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:59.734631 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g72sn_2b9ad799-4001-448a-8947-36def65e3f94/dns/0.log" Jul 14 10:41:59.756445 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:59.756423 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g72sn_2b9ad799-4001-448a-8947-36def65e3f94/kube-rbac-proxy/0.log" Jul 14 10:41:59.881802 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:41:59.881784 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vm4nd_71976dd2-2c7d-406a-8e1a-ffc873e42174/dns-node-resolver/0.log" Jul 14 10:42:00.560168 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:00.560131 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-khgqj_4ee471c5-e56c-45c5-b717-c690e2d36229/node-ca/0.log" Jul 14 10:42:01.502712 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:01.502683 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gzbt4_29b302a7-97d9-44b4-b065-cf6c43bf330f/istio-proxy/0.log" Jul 14 10:42:01.606010 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:01.605983 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-45qvb_6051ad4d-18c2-4c64-80e4-998423db2c9b/discovery/0.log" Jul 14 10:42:01.687009 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:01.686988 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/0.log" Jul 14 10:42:01.712183 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:01.712140 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/1.log" Jul 14 10:42:02.554068 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:02.554040 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-chm6b_c8013d14-4407-485c-8817-c2bb32a9a7a0/serve-healthcheck-canary/0.log" Jul 14 10:42:03.263297 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:03.263266 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwk7v_8b0e354f-e5d2-4f1b-9324-2912327875b1/kube-rbac-proxy/0.log" Jul 14 10:42:03.285546 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:03.285524 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwk7v_8b0e354f-e5d2-4f1b-9324-2912327875b1/exporter/0.log" Jul 14 10:42:03.307692 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:03.307672 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bwk7v_8b0e354f-e5d2-4f1b-9324-2912327875b1/extractor/0.log" Jul 14 10:42:04.356828 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:04.356801 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5rfhj/perf-node-gather-daemonset-r6kd8" Jul 14 10:42:05.888612 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:05.888590 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29733735-k6mfs_84f9229e-545c-48be-a7e9-ed5620b55bf7/cleanup/0.log" Jul 14 10:42:05.915809 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:05.915784 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29733750-s5wr8_f1de1108-9f98-42fb-8ec1-07a9eabe2ace/cleanup/0.log" Jul 14 10:42:06.016466 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:06.016438 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-667d7fccf6-k9rsj_b567d770-0548-448a-bdc1-4b03840a7869/manager/0.log" Jul 14 10:42:06.329926 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:06.329900 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj_e69b7b2c-d086-489a-b005-fa5ba1fdfff6/manager/0.log" Jul 14 10:42:06.351240 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:06.351215 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b96c8bfcf-pdhrj_e69b7b2c-d086-489a-b005-fa5ba1fdfff6/copy-manifests/0.log" Jul 14 10:42:07.297371 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.297342 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gzbt4_29b302a7-97d9-44b4-b065-cf6c43bf330f/istio-proxy/0.log" Jul 14 10:42:07.401290 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.401260 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-45qvb_6051ad4d-18c2-4c64-80e4-998423db2c9b/discovery/0.log" Jul 14 10:42:07.449587 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.449550 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/0.log" Jul 14 10:42:07.474213 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.474186 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/1.log" Jul 14 10:42:07.821852 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.821818 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-589999c949-twspv_8e1f51cb-c70a-40bf-8d6c-793ccc477345/manager/0.log" Jul 14 10:42:07.856748 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:07.856728 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-j99st_ae11fcad-1c6a-4879-95bb-becff826d505/openshift-lws-operator/0.log" Jul 14 10:42:08.809817 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:08.809730 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gzbt4_29b302a7-97d9-44b4-b065-cf6c43bf330f/istio-proxy/0.log" Jul 14 10:42:08.912295 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:08.912264 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-45qvb_6051ad4d-18c2-4c64-80e4-998423db2c9b/discovery/0.log" Jul 14 10:42:08.960273 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:08.960241 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/0.log" Jul 14 10:42:08.988282 ip-10-0-141-78 kubenswrapper[2569]: I0714 10:42:08.988254 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-btd4c_fdce0995-348a-4885-825f-b040a0f4001c/istio-proxy/1.log"