Jul 23 19:00:52.314125 ip-10-0-139-123 systemd[1]: Starting Kubernetes Kubelet... Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 19:00:52.753348 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.748791 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753567 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753580 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753584 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753587 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753590 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753592 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753595 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753597 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753600 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753603 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753605 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753608 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753611 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753614 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753617 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753620 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753623 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753674 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753989 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:52.797702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.753999 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754004 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754009 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754014 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754018 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754024 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754029 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754034 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754038 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754042 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754047 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754056 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754061 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754065 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754071 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754076 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754080 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754085 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754090 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754094 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:52.816749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754099 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754103 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754107 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754112 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754120 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754124 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754130 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754137 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754141 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754156 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754160 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754210 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754216 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754221 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754226 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754237 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754242 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754247 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754251 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:52.910159 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754255 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:52.858496 ip-10-0-139-123 systemd[1]: Started Kubernetes Kubelet. Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754260 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754264 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754268 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754272 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754276 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754280 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754285 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754290 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754298 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754302 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754309 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754313 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754317 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754321 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754325 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754330 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754334 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754339 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754343 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.108702 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754352 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754357 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754360 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754363 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754366 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754370 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754373 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754378 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754382 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754385 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754388 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754393 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754398 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754404 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754408 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754411 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754414 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754417 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754420 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.111943 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754423 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.754426 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756029 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756038 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756042 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756045 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756048 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756055 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756058 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756061 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756063 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756066 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756069 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756072 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.756075 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757778 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757785 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757788 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757792 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757800 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:53.113215 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757804 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757809 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757849 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757872 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757877 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757902 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757921 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757927 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757931 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757934 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757937 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757941 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757945 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757948 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757954 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757957 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757960 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757977 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757984 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:53.114557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757988 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757994 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.757998 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758001 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758005 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758007 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758010 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758013 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758016 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758019 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758024 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758027 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758030 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758033 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758036 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758039 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758042 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758045 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758048 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758050 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.115475 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758053 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758057 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758061 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758067 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758070 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758074 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758077 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758080 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758084 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758089 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758093 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758096 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758102 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758107 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758109 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758115 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758118 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758120 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758123 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758126 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.116398 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758128 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758131 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758134 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758136 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758139 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758142 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758145 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758147 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758152 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758155 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758160 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758163 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758165 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758168 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758172 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758176 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758179 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758182 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758184 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758190 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:53.117338 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758193 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.758195 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763468 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763477 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763483 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763489 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763492 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763495 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763498 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763500 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763503 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763506 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763508 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763511 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763513 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763516 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763519 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763523 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763526 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:53.118249 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763530 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763535 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763539 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763542 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763545 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763549 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763552 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763556 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763558 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763561 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763563 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763566 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763568 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763571 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763573 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763576 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763579 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763583 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763587 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763589 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:53.119192 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763592 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763595 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763597 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763600 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763602 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763605 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763607 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763610 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763612 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763615 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763617 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763619 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763622 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763624 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763627 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763629 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763632 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763635 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763638 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:53.120130 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763641 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763643 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763645 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763648 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763650 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763653 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763655 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763658 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763660 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763663 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763666 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763670 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763673 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763676 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763678 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763681 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763683 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763686 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763688 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763691 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.121103 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763693 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763696 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763698 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763703 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763706 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763709 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763712 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763716 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763719 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763722 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763724 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763727 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763729 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763731 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763734 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763736 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763739 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763741 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763744 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.121942 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763746 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763749 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763751 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.763755 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763850 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763861 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763867 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763872 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763877 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763880 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763884 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763888 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763891 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763895 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763898 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763901 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763905 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763908 2576 flags.go:64] FLAG: --cgroup-root="" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763911 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763913 2576 flags.go:64] FLAG: --client-ca-file="" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763916 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763919 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763924 2576 flags.go:64] FLAG: --cluster-domain="" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763927 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 19:00:53.122884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763930 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763933 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763936 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763939 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763943 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763946 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763949 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763952 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763955 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763958 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763961 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763966 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763972 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763976 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763979 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763982 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763985 2576 flags.go:64] FLAG: --enable-server="true" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763988 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763992 2576 flags.go:64] FLAG: --event-burst="100" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763995 2576 flags.go:64] FLAG: --event-qps="50" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.763999 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764002 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764004 2576 flags.go:64] FLAG: --eviction-hard="" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764008 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 19:00:53.123825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764011 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764014 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764017 2576 flags.go:64] FLAG: --eviction-soft="" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764020 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764023 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764026 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764029 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764032 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764034 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764037 2576 flags.go:64] FLAG: --feature-gates="" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764041 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764044 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764047 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764051 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764053 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764056 2576 flags.go:64] FLAG: --help="false" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764059 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-139-123.ec2.internal" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764062 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764065 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764069 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764074 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764077 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764081 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 19:00:53.124772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764084 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764087 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764090 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764092 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764096 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764099 2576 flags.go:64] FLAG: --kube-reserved="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764102 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764104 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764107 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764110 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764113 2576 flags.go:64] FLAG: --lock-file="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764116 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764119 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764121 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764127 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764130 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764132 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764135 2576 flags.go:64] FLAG: --logging-format="text" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764138 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764141 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764144 2576 flags.go:64] FLAG: --manifest-url="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764147 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764151 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764154 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764158 2576 flags.go:64] FLAG: --max-pods="110" Jul 23 19:00:53.126346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764161 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764164 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764167 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764171 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764177 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764179 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764182 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764190 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764193 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764196 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764199 2576 flags.go:64] FLAG: --pod-cidr="" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764202 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764208 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764211 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764214 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764217 2576 flags.go:64] FLAG: --port="10250" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764220 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764223 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fd29d22bdb5d7ef5" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764226 2576 flags.go:64] FLAG: --qos-reserved="" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764229 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764231 2576 flags.go:64] FLAG: --register-node="true" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764234 2576 flags.go:64] FLAG: --register-with-taints="" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764238 2576 flags.go:64] FLAG: --registry-burst="10" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764241 2576 flags.go:64] FLAG: --registry-qps="5" Jul 23 19:00:53.127740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764243 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764246 2576 flags.go:64] FLAG: --reserved-memory="" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764250 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764253 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764256 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764259 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764262 2576 flags.go:64] FLAG: --runonce="false" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764264 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764267 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764270 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764273 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764276 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764282 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764285 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764288 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764291 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764294 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764297 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764300 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764303 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764306 2576 flags.go:64] FLAG: --system-cgroups="" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764309 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764315 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764317 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764320 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 19:00:53.128665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764324 2576 flags.go:64] FLAG: --tls-min-version="" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764327 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764329 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764333 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764336 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764339 2576 flags.go:64] FLAG: --v="2" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764347 2576 flags.go:64] FLAG: --version="false" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764352 2576 flags.go:64] FLAG: --vmodule="" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764356 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764359 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764522 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764526 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764529 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764532 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764535 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764537 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764541 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764544 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764547 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764552 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764555 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764558 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.129632 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764560 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764563 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764567 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764569 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764572 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764576 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764580 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764583 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764586 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764590 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764593 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764595 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764598 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764601 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764603 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764606 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764608 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764611 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764613 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.134717 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764616 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764619 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764621 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764623 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764626 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764628 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764631 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764633 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764635 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764638 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764644 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764646 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764649 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764652 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764654 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764657 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764660 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764663 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764665 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764668 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:53.135618 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764670 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764673 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764676 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764678 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764681 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764683 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764686 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764688 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764691 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764693 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764696 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764698 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764701 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764704 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764706 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764709 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764712 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764714 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764717 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764719 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.255442 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764722 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764724 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764729 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764732 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764735 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764737 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764740 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764742 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764745 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764748 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764750 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764753 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764755 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764758 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764760 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764762 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764765 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764767 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764770 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764772 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:53.256745 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764775 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764777 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764780 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764782 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764785 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764787 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764789 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.764792 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.764805 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.771937 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.771953 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.771972 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.771978 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772034 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772040 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.257636 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772043 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772047 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772049 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772052 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772056 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772058 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772062 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772065 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772068 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772070 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772074 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772076 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772079 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772082 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772085 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772088 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772091 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772093 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772096 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772099 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:53.258564 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772102 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772104 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772108 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772112 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772115 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772119 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772122 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772127 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772130 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772133 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772136 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772138 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772141 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772144 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772146 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772149 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772152 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772155 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772157 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:53.259431 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772161 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772164 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772166 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772169 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772172 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772174 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772177 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772180 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772182 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772185 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772187 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772190 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772194 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772197 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772200 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772203 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772206 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772209 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772212 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.260325 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772214 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772218 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772221 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772224 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772226 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772229 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772231 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772234 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772236 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772239 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772243 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772245 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772248 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772251 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772254 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772257 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772259 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772262 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772264 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:53.261183 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772267 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772270 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772273 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772276 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772279 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772281 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772284 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772286 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772289 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772291 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772294 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772296 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772299 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772301 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772305 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772308 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772310 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772313 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772315 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.262062 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772318 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.772323 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772437 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772441 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772446 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772449 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772451 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772455 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772457 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772461 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772463 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772466 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772469 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772471 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772474 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 19:00:53.262979 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772476 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772479 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772482 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772484 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772487 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772489 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772492 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772494 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772497 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772499 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772502 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772505 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772507 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772511 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772513 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772516 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772518 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772521 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772523 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 19:00:53.264376 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772526 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772529 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772532 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772535 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772538 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772540 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772543 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772545 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772550 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772553 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772555 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772558 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772562 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772565 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772568 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772572 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772574 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772578 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772580 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 19:00:53.265218 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772583 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772586 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772588 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772591 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772593 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772596 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772599 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772603 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772606 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772608 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772611 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772614 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772616 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772619 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772621 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772625 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772628 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772631 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772633 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772635 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 19:00:53.266129 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772638 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772640 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772644 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772646 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772648 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772651 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772654 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772656 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772659 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772661 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772664 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772666 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772669 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772671 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772674 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772676 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772679 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772682 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772685 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 19:00:53.267006 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772690 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772692 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772695 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772697 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772700 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772702 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772705 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772707 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:52.772711 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.772716 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.772848 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.775312 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.777041 2576 server.go:1019] "Starting client certificate rotation" Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.777136 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 19:00:53.267918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.777189 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.780073 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.781557 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.799253 2576 log.go:25] "Validated CRI v1 runtime API" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.817799 2576 log.go:25] "Validated CRI v1 image API" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.818735 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.821555 2576 fs.go:135] Filesystem UUIDs: map[10fc471e-5e17-4339-9527-047b4e449e0b:/dev/nvme0n1p3 556b6b43-59fa-4ce7-a6db-7001a834d554:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.821576 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 19:00:53.268368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.831489 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.831899 2576 manager.go:216] Machine: {Timestamp:2026-07-23 19:00:52.829220661 +0000 UTC m=+0.407537409 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099615 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20d5a4940a2f916c9dc4125874e81d SystemUUID:ec20d5a4-940a-2f91-6c9d-c4125874e81d BootID:0df0280a-288d-4b66-8ab2-dc80556a7fcd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:cb:0a:e8:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:cb:0a:e8:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:7a:42:ef:89:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.832038 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.832151 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.836008 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.836052 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.837654 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.837670 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.837680 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.838524 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.840430 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844448 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844593 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844622 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844636 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844652 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844664 2576 kubelet.go:398] "Adding apiserver pod source" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.844675 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 19:00:53.269274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.845129 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.845147 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.847591 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.850011 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.850039 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851450 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851468 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851480 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851489 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851501 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851509 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851520 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851528 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851537 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851545 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851563 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.851571 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.853316 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.853330 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.855693 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.855746 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.855924 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.857713 2576 server.go:1268] "Started kubelet" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.858214 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.860307 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.868113 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jp6tj" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.874345 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.877261 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.880815 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jp6tj" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.880686 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.881164 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.881421 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.881499 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.882540 2576 factory.go:55] Registering systemd factory Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.882564 2576 factory.go:223] Registration of the systemd container factory successfully Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.882623 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.882778 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883091 2576 factory.go:153] Registering CRI-O factory Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883105 2576 factory.go:223] Registration of the crio container factory successfully Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883154 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883178 2576 factory.go:103] Registering Raw factory Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883193 2576 manager.go:1196] Started watching for new ooms in manager Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.883702 2576 manager.go:349] Starting recovery of all containers Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.885738 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.885785 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.886009 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.886020 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.897734 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:53.270148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.900122 2576 manager.go:354] Recovery completed Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.903693 2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-123.ec2.internal\" not found" node="ip-10-0-139-123.ec2.internal" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.908330 2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.911423 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.913260 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientMemory" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.913294 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.913311 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientPID" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.913989 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914002 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914022 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914023 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914749 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientMemory" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914780 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.914796 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientPID" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.915369 2576 policy_none.go:49] "None policy: Start" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.915384 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.915398 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:52.916324 2576 policy_none.go:47] "Start" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:52.981918 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.039609 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.040813 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.040864 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.040887 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.040929 2576 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.043732 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.082852 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.141866 2576 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 23 19:00:53.271478 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.183008 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.388038 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.283548 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.388038 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.342800 2576 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 23 19:00:53.388038 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.384161 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.509190 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.484616 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.614366 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.585061 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.727793 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.685586 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.846576 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.743871 2576 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 23 19:00:53.846576 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.776019 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 19:00:53.846576 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.776174 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 19:00:53.846576 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.776209 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 19:00:53.846576 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.786402 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.961505 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.887402 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:53.961505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.887399 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 18:55:53 +0000 UTC" deadline="2028-01-25 01:13:08.78141036 +0000 UTC" Jul 23 19:00:53.961505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:53.887427 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13206h12m14.893987256s" Jul 23 19:00:54.102246 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:53.987527 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.102246 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.088019 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.235142 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.188945 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.373461 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.289921 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.392024 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.390820 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.516460 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.491508 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.659101 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.544943 2576 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 23 19:00:54.659101 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.592272 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.799643 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.693003 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.799643 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.793936 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:54.931557 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.894039 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:55.051974 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:54.994950 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:55.169436 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:55.095528 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:55.196274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:55.196203 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:55.200934 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.200641 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:55.202699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.201503 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:55.203031 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.202796 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:55.204446 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.204419 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 19:00:55.207255 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.207232 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-123.ec2.internal" not found Jul 23 19:00:55.209802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.209790 2576 manager.go:337] "Starting Device Plugin manager" Jul 23 19:00:55.209895 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:55.209879 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 19:00:55.209965 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.209897 2576 server.go:87] "Starting device plugin registration server" Jul 23 19:00:55.210261 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.210239 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 19:00:55.210332 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.210257 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 19:00:55.210398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.210385 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 19:00:55.210473 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.210463 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 19:00:55.210517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.210475 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 19:00:55.211404 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:55.211383 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 19:00:55.211503 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:55.211430 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-123.ec2.internal\" not found" Jul 23 19:00:55.225041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.225020 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-123.ec2.internal" not found Jul 23 19:00:55.286580 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.286558 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-123.ec2.internal" not found Jul 23 19:00:55.311432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.311401 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 19:00:55.312476 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.312461 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientMemory" Jul 23 19:00:55.312547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.312491 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 19:00:55.312547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.312503 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeHasSufficientPID" Jul 23 19:00:55.312547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.312528 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-123.ec2.internal" Jul 23 19:00:55.325041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.325024 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-123.ec2.internal" Jul 23 19:00:55.848065 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.848031 2576 apiserver.go:52] "Watching apiserver" Jul 23 19:00:55.854615 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.854593 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 19:00:55.876468 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.876444 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 19:00:55.895914 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.895890 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 19:00:55.954301 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.954273 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7s9tc" Jul 23 19:00:55.964283 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:55.964260 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7s9tc" Jul 23 19:00:56.148227 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.148106 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal"] Jul 23 19:00:56.150331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.150315 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.151907 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.151883 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-lwp75","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj","openshift-cluster-node-tuning-operator/tuned-gqpfc","openshift-network-diagnostics/network-check-target-j6lsv","kube-system/global-pull-secret-syncer-2q6nq","openshift-dns/node-resolver-ct8h8","openshift-image-registry/node-ca-4xq9v","openshift-multus/multus-6s2m7","openshift-multus/multus-additional-cni-plugins-p44x8","openshift-multus/network-metrics-daemon-kvmj8","openshift-network-operator/iptables-alerter-kjv4k","openshift-ovn-kubernetes/ovnkube-node-cgsqw"] Jul 23 19:00:56.152016 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.151939 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.154374 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.154360 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.156242 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.156225 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.157140 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.157123 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.157234 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.157215 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.157375 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.157360 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t7ljg\"" Jul 23 19:00:56.158236 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.158222 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.158811 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.158510 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.159172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.159119 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nq9pp\"" Jul 23 19:00:56.159279 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.159224 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.160640 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.160275 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.160640 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.160390 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.160640 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.160636 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 19:00:56.160872 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.160792 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hqxvj\"" Jul 23 19:00:56.160926 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.160900 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.162633 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.162617 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.162700 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.162682 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:00:56.163879 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.163859 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.163977 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.163859 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.164313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.164295 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 19:00:56.164364 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.164348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jhpsj\"" Jul 23 19:00:56.164364 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.164356 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 19:00:56.164627 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.164612 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.166766 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.166749 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.167810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.167795 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.168086 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.168063 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 19:00:56.168175 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.168114 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.168708 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.168693 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jgngt\"" Jul 23 19:00:56.168912 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.168892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.170762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.170745 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.170762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.170754 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.170911 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.170779 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 19:00:56.170911 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.170888 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rnrlk\"" Jul 23 19:00:56.171209 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.171196 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.171657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.171641 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 19:00:56.171728 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.171645 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-265hr\"" Jul 23 19:00:56.171848 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.171832 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 19:00:56.172072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.172058 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.173801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.173787 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:56.173924 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.173899 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:00:56.174140 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174124 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 19:00:56.174403 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174379 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 19:00:56.174403 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 19:00:56.174561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174389 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 19:00:56.174561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174438 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 19:00:56.174561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174454 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 19:00:56.174561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.174559 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p6cd8\"" Jul 23 19:00:56.176483 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.176463 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.176576 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.176526 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:00:56.178801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.178784 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.181517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.181501 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 19:00:56.181591 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.181505 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fj7gq\"" Jul 23 19:00:56.181591 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.181507 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 19:00:56.184346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.184333 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 19:00:56.187486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.187474 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.191116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.191095 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 19:00:56.191463 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.191446 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal"] Jul 23 19:00:56.197199 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.197183 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal"] Jul 23 19:00:56.197548 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.197534 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 19:00:56.207193 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207176 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-etc-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207201 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-log-socket\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207220 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-modprobe-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.207260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-lib-modules\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a956c0a0-2420-4040-913e-d714b4dc5418-tmp-dir\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74xp9\" (UniqueName: \"kubernetes.io/projected/68a4014d-9b37-458d-b82d-4a29ef017488-kube-api-access-74xp9\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-kubelet-config\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-dbus\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a63dd2beb5b4bbbe932b9b8b6353efdf-config\") pod \"kube-apiserver-proxy-ip-10-0-139-123.ec2.internal\" (UID: \"a63dd2beb5b4bbbe932b9b8b6353efdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-netd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207365 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmlqz\" (UniqueName: \"kubernetes.io/projected/922464fd-cb2e-4510-b706-39615e4216bf-kube-api-access-mmlqz\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207380 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-systemd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5g4j\" (UniqueName: \"kubernetes.io/projected/0e92e899-d841-48d9-aefd-9492aa8784a0-kube-api-access-f5g4j\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-socket-dir-parent\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207442 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-multus-certs\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207457 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207486 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-987pf\" (UniqueName: \"kubernetes.io/projected/3f45f70b-fce7-4786-9a4b-9661f273fa8e-kube-api-access-987pf\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207501 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a956c0a0-2420-4040-913e-d714b4dc5418-hosts-file\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207515 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f9582889-69da-4fc1-9340-3998fd371a39-host\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-cni-binary-copy\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-kubelet\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-hostroot\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8b9m\" (UniqueName: \"kubernetes.io/projected/5bb7b0bd-5164-4341-8948-08dd57301b5b-kube-api-access-h8b9m\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-bin\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-daemon-config\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-env-overrides\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207677 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-sys-fs\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207705 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-os-release\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207720 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj64l\" (UniqueName: \"kubernetes.io/projected/f9582889-69da-4fc1-9340-3998fd371a39-kube-api-access-qj64l\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.207979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-os-release\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-k8s-cni-cncf-io\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207768 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0c70287-dd25-4cd7-b656-93a14d7605ef-konnectivity-ca\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207791 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-registration-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-conf\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-cnibin\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207838 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l6s8\" (UniqueName: \"kubernetes.io/projected/892756f1-263f-46af-b14f-a38baa295c80-kube-api-access-7l6s8\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207854 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-var-lib-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207916 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-cnibin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207931 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzbc\" (UniqueName: \"kubernetes.io/projected/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-kube-api-access-5mzbc\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-config\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.207986 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-run\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208002 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-system-cni-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.208691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-bin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208032 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-multus\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-var-lib-kubelet\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-tmp\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-etc-kubernetes\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208104 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-kubelet\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-systemd-units\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208134 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-netns\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208148 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovn-node-metrics-cert\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-script-lib\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-kubernetes\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208228 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0c70287-dd25-4cd7-b656-93a14d7605ef-agent-certs\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.209398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208257 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208272 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-device-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysconfig\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-netns\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-conf-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208333 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-host-slash\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-node-log\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208377 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-676sh\" (UniqueName: \"kubernetes.io/projected/a956c0a0-2420-4040-913e-d714b4dc5418-kube-api-access-676sh\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-system-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-systemd\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208429 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-sys\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208461 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-ovn\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-socket-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.210088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-host\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.210831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-etc-tuned\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.210831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f9582889-69da-4fc1-9340-3998fd371a39-serviceca\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.210831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-iptables-alerter-script\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.210831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.208569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-slash\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.308968 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.308945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.309061 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.308975 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-os-release\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309061 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.308993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj64l\" (UniqueName: \"kubernetes.io/projected/f9582889-69da-4fc1-9340-3998fd371a39-kube-api-access-qj64l\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.309061 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309032 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-os-release\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-k8s-cni-cncf-io\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0c70287-dd25-4cd7-b656-93a14d7605ef-konnectivity-ca\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309082 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-os-release\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309155 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-k8s-cni-cncf-io\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309158 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-registration-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309168 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-os-release\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309198 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-conf\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.309230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309213 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-registration-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309224 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-cnibin\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309272 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l6s8\" (UniqueName: \"kubernetes.io/projected/892756f1-263f-46af-b14f-a38baa295c80-kube-api-access-7l6s8\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309249 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-cnibin\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.309279 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309316 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-var-lib-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-var-lib-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.309345 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:56.809323066 +0000 UTC m=+4.387639816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309373 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-conf\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309412 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309447 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-cnibin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzbc\" (UniqueName: \"kubernetes.io/projected/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-kube-api-access-5mzbc\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309479 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-config\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309594 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309624 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-cnibin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.309648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-run\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309667 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-system-cni-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309691 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-bin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-multus\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-var-lib-kubelet\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309747 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysctl-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309759 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-tmp\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309762 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-run\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-etc-kubernetes\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309786 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-multus\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-kubelet\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-systemd-units\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309814 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-system-cni-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309823 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309830 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-netns\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309870 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-etc-kubernetes\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309878 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-cni-bin\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.310354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309897 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovn-node-metrics-cert\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309920 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-script-lib\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-kubernetes\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309987 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0c70287-dd25-4cd7-b656-93a14d7605ef-agent-certs\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-device-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysconfig\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310163 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-netns\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-conf-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-host-slash\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310237 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-node-log\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310276 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-676sh\" (UniqueName: \"kubernetes.io/projected/a956c0a0-2420-4040-913e-d714b4dc5418-kube-api-access-676sh\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-system-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.311247 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-systemd\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310385 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-sys\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-ovn\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310507 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310510 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-socket-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310563 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-host\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310586 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-etc-tuned\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310603 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f9582889-69da-4fc1-9340-3998fd371a39-serviceca\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310610 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0c70287-dd25-4cd7-b656-93a14d7605ef-konnectivity-ca\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310617 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-socket-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310628 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310612 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310689 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-netns\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310690 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-kubernetes\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309962 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-systemd-units\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310788 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-system-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.312123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.309936 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-kubelet\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310892 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-conf-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310977 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-host-slash\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311054 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f9582889-69da-4fc1-9340-3998fd371a39-serviceca\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311106 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-node-log\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311142 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-host\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311173 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-cni-dir\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311189 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-iptables-alerter-script\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311210 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-script-lib\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311192 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-device-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.310620 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-iptables-alerter-script\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311230 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-sysconfig\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311251 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311287 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-var-lib-kubelet\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.311321 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:56.313015 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311359 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-netns\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.311383 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:56.811364344 +0000 UTC m=+4.389681095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311399 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-ovn\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311397 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-systemd\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311441 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-sys\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311448 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5e67c885e56ac5079a5f283c430044-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal\" (UID: \"3f5e67c885e56ac5079a5f283c430044\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-slash\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-etc-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311560 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovnkube-config\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311574 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-etc-openvswitch\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311564 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-slash\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-log-socket\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311630 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-modprobe-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311721 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311709 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-lib-modules\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.313861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311765 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a956c0a0-2420-4040-913e-d714b4dc5418-tmp-dir\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311769 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-log-socket\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311793 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74xp9\" (UniqueName: \"kubernetes.io/projected/68a4014d-9b37-458d-b82d-4a29ef017488-kube-api-access-74xp9\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311805 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-lib-modules\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311821 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-kubelet-config\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311862 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-dbus\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311878 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/922464fd-cb2e-4510-b706-39615e4216bf-etc-modprobe-d\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a63dd2beb5b4bbbe932b9b8b6353efdf-config\") pod \"kube-apiserver-proxy-ip-10-0-139-123.ec2.internal\" (UID: \"a63dd2beb5b4bbbe932b9b8b6353efdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-netd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311939 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-kubelet-config\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmlqz\" (UniqueName: \"kubernetes.io/projected/922464fd-cb2e-4510-b706-39615e4216bf-kube-api-access-mmlqz\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312066 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a956c0a0-2420-4040-913e-d714b4dc5418-tmp-dir\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312086 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-systemd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312040 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-netd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312116 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5g4j\" (UniqueName: \"kubernetes.io/projected/0e92e899-d841-48d9-aefd-9492aa8784a0-kube-api-access-f5g4j\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-socket-dir-parent\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.314682 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312158 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-run-systemd\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-multus-certs\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312212 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-987pf\" (UniqueName: \"kubernetes.io/projected/3f45f70b-fce7-4786-9a4b-9661f273fa8e-kube-api-access-987pf\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a956c0a0-2420-4040-913e-d714b4dc5418-hosts-file\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-socket-dir-parent\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312290 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312324 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f9582889-69da-4fc1-9340-3998fd371a39-host\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312346 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-run-multus-certs\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-cni-binary-copy\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312047 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c29a651-9e48-4ff9-842c-ccd602552695-dbus\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312386 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-kubelet\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312413 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-hostroot\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312438 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8b9m\" (UniqueName: \"kubernetes.io/projected/5bb7b0bd-5164-4341-8948-08dd57301b5b-kube-api-access-h8b9m\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312464 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-bin\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-daemon-config\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312523 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-env-overrides\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312534 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/892756f1-263f-46af-b14f-a38baa295c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.315284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.311983 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a63dd2beb5b4bbbe932b9b8b6353efdf-config\") pod \"kube-apiserver-proxy-ip-10-0-139-123.ec2.internal\" (UID: \"a63dd2beb5b4bbbe932b9b8b6353efdf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-sys-fs\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312577 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/892756f1-263f-46af-b14f-a38baa295c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312598 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0e92e899-d841-48d9-aefd-9492aa8784a0-sys-fs\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312641 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f9582889-69da-4fc1-9340-3998fd371a39-host\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312643 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-hostroot\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312840 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a956c0a0-2420-4040-913e-d714b4dc5418-hosts-file\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312851 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f45f70b-fce7-4786-9a4b-9661f273fa8e-host-cni-bin\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.312904 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7b0bd-5164-4341-8948-08dd57301b5b-host-var-lib-kubelet\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.313210 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-multus-daemon-config\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.313279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f45f70b-fce7-4786-9a4b-9661f273fa8e-env-overrides\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.313320 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bb7b0bd-5164-4341-8948-08dd57301b5b-cni-binary-copy\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.314782 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-etc-tuned\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.314824 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/922464fd-cb2e-4510-b706-39615e4216bf-tmp\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.314950 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0c70287-dd25-4cd7-b656-93a14d7605ef-agent-certs\") pod \"konnectivity-agent-lwp75\" (UID: \"a0c70287-dd25-4cd7-b656-93a14d7605ef\") " pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.315782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.315098 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f45f70b-fce7-4786-9a4b-9661f273fa8e-ovn-node-metrics-cert\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.325097 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.325071 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj64l\" (UniqueName: \"kubernetes.io/projected/f9582889-69da-4fc1-9340-3998fd371a39-kube-api-access-qj64l\") pod \"node-ca-4xq9v\" (UID: \"f9582889-69da-4fc1-9340-3998fd371a39\") " pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.341523 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.341504 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:00:56.341602 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.341529 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:00:56.341602 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.341545 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:56.341666 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.341616 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:56.841600226 +0000 UTC m=+4.419916982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:56.343067 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.343046 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l6s8\" (UniqueName: \"kubernetes.io/projected/892756f1-263f-46af-b14f-a38baa295c80-kube-api-access-7l6s8\") pod \"multus-additional-cni-plugins-p44x8\" (UID: \"892756f1-263f-46af-b14f-a38baa295c80\") " pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.343520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.343495 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-987pf\" (UniqueName: \"kubernetes.io/projected/3f45f70b-fce7-4786-9a4b-9661f273fa8e-kube-api-access-987pf\") pod \"ovnkube-node-cgsqw\" (UID: \"3f45f70b-fce7-4786-9a4b-9661f273fa8e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.346223 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.346195 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8b9m\" (UniqueName: \"kubernetes.io/projected/5bb7b0bd-5164-4341-8948-08dd57301b5b-kube-api-access-h8b9m\") pod \"multus-6s2m7\" (UID: \"5bb7b0bd-5164-4341-8948-08dd57301b5b\") " pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.346573 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.346557 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-676sh\" (UniqueName: \"kubernetes.io/projected/a956c0a0-2420-4040-913e-d714b4dc5418-kube-api-access-676sh\") pod \"node-resolver-ct8h8\" (UID: \"a956c0a0-2420-4040-913e-d714b4dc5418\") " pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.350830 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.350804 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p44x8" Jul 23 19:00:56.352379 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.352362 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74xp9\" (UniqueName: \"kubernetes.io/projected/68a4014d-9b37-458d-b82d-4a29ef017488-kube-api-access-74xp9\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.353677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.353656 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmlqz\" (UniqueName: \"kubernetes.io/projected/922464fd-cb2e-4510-b706-39615e4216bf-kube-api-access-mmlqz\") pod \"tuned-gqpfc\" (UID: \"922464fd-cb2e-4510-b706-39615e4216bf\") " pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.355393 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.355370 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzbc\" (UniqueName: \"kubernetes.io/projected/73d1b014-17b7-4ec0-9af9-aa8b5beda1be-kube-api-access-5mzbc\") pod \"iptables-alerter-kjv4k\" (UID: \"73d1b014-17b7-4ec0-9af9-aa8b5beda1be\") " pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.357241 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.357198 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5g4j\" (UniqueName: \"kubernetes.io/projected/0e92e899-d841-48d9-aefd-9492aa8784a0-kube-api-access-f5g4j\") pod \"aws-ebs-csi-driver-node-7ssfj\" (UID: \"0e92e899-d841-48d9-aefd-9492aa8784a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.494092 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.494013 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.500222 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.500201 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" Jul 23 19:00:56.517855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.517824 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" Jul 23 19:00:56.534693 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.534667 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ct8h8" Jul 23 19:00:56.551634 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.551603 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4xq9v" Jul 23 19:00:56.569103 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.569079 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6s2m7" Jul 23 19:00:56.584925 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.584902 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjv4k" Jul 23 19:00:56.600201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.600178 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:00:56.614965 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.614948 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:00:56.640740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.640712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" Jul 23 19:00:56.817119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.817029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:56.817119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.817096 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:56.817318 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.817176 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:56.817318 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.817197 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:56.817318 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.817257 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:57.817234096 +0000 UTC m=+5.395550846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:56.817318 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.817279 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:57.817268991 +0000 UTC m=+5.395585726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:56.918421 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.918391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:56.918588 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.918517 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:00:56.918588 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.918532 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:00:56.918588 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.918541 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:56.918588 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:56.918587 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:57.91857388 +0000 UTC m=+5.496890616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:56.969231 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.969178 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:55:56 +0000 UTC" deadline="2028-04-16 22:37:30.048586824 +0000 UTC" Jul 23 19:00:56.969231 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:56.969224 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15195h36m33.079365062s" Jul 23 19:00:57.191188 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.191147 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f45f70b_fce7_4786_9a4b_9661f273fa8e.slice/crio-e4dd38e8cc1fe856a48145cc896f579ee44586d19648b858b7f03eed0ab542fb WatchSource:0}: Error finding container e4dd38e8cc1fe856a48145cc896f579ee44586d19648b858b7f03eed0ab542fb: Status 404 returned error can't find the container with id e4dd38e8cc1fe856a48145cc896f579ee44586d19648b858b7f03eed0ab542fb Jul 23 19:00:57.201203 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.201181 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 19:00:57.203290 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.203266 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb7b0bd_5164_4341_8948_08dd57301b5b.slice/crio-0bd09564a3225df52cd51f4c75d6ccdced4cd5b590002f294e0901b17fe3998c WatchSource:0}: Error finding container 0bd09564a3225df52cd51f4c75d6ccdced4cd5b590002f294e0901b17fe3998c: Status 404 returned error can't find the container with id 0bd09564a3225df52cd51f4c75d6ccdced4cd5b590002f294e0901b17fe3998c Jul 23 19:00:57.221245 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.221209 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922464fd_cb2e_4510_b706_39615e4216bf.slice/crio-029cfaba6304adbab9bb2e498536bc32514290aa44e796b39243c44219a2986c WatchSource:0}: Error finding container 029cfaba6304adbab9bb2e498536bc32514290aa44e796b39243c44219a2986c: Status 404 returned error can't find the container with id 029cfaba6304adbab9bb2e498536bc32514290aa44e796b39243c44219a2986c Jul 23 19:00:57.235670 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.235646 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod892756f1_263f_46af_b14f_a38baa295c80.slice/crio-8936e31457ac40ffd9cf5fd421c58e9b18c768b3cbd427076a25f289493d3337 WatchSource:0}: Error finding container 8936e31457ac40ffd9cf5fd421c58e9b18c768b3cbd427076a25f289493d3337: Status 404 returned error can't find the container with id 8936e31457ac40ffd9cf5fd421c58e9b18c768b3cbd427076a25f289493d3337 Jul 23 19:00:57.244008 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.243988 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c70287_dd25_4cd7_b656_93a14d7605ef.slice/crio-a3dcc3a3e6748a56584ec4a0e0c8179bcd674540fbb82a4f583f971e68c49b3f WatchSource:0}: Error finding container a3dcc3a3e6748a56584ec4a0e0c8179bcd674540fbb82a4f583f971e68c49b3f: Status 404 returned error can't find the container with id a3dcc3a3e6748a56584ec4a0e0c8179bcd674540fbb82a4f583f971e68c49b3f Jul 23 19:00:57.252372 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.252354 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63dd2beb5b4bbbe932b9b8b6353efdf.slice/crio-fad3310c67be3dd818b4ee24a004ad86a253c0be7dd7dc780a285d2f6176c751 WatchSource:0}: Error finding container fad3310c67be3dd818b4ee24a004ad86a253c0be7dd7dc780a285d2f6176c751: Status 404 returned error can't find the container with id fad3310c67be3dd818b4ee24a004ad86a253c0be7dd7dc780a285d2f6176c751 Jul 23 19:00:57.253116 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.253092 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda956c0a0_2420_4040_913e_d714b4dc5418.slice/crio-96ded8e238693577b5c2d8d77ff10c74546e8d145bf25654a48fc2a859deefa2 WatchSource:0}: Error finding container 96ded8e238693577b5c2d8d77ff10c74546e8d145bf25654a48fc2a859deefa2: Status 404 returned error can't find the container with id 96ded8e238693577b5c2d8d77ff10c74546e8d145bf25654a48fc2a859deefa2 Jul 23 19:00:57.263529 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.263487 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e92e899_d841_48d9_aefd_9492aa8784a0.slice/crio-ba4210ae5ce92293ef904836cf83586ddfa9e14f5e9d6029a290b7f53d9d6480 WatchSource:0}: Error finding container ba4210ae5ce92293ef904836cf83586ddfa9e14f5e9d6029a290b7f53d9d6480: Status 404 returned error can't find the container with id ba4210ae5ce92293ef904836cf83586ddfa9e14f5e9d6029a290b7f53d9d6480 Jul 23 19:00:57.276516 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.276494 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5e67c885e56ac5079a5f283c430044.slice/crio-6084ee056209f0966bcf6e24a6ef36847fca17f91e89beefcc255299a24d144d WatchSource:0}: Error finding container 6084ee056209f0966bcf6e24a6ef36847fca17f91e89beefcc255299a24d144d: Status 404 returned error can't find the container with id 6084ee056209f0966bcf6e24a6ef36847fca17f91e89beefcc255299a24d144d Jul 23 19:00:57.408639 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.408604 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9582889_69da_4fc1_9340_3998fd371a39.slice/crio-ad0d8639da933ef07c36d0305bba94c84c727e2aeb325a230765a3645ccbfc29 WatchSource:0}: Error finding container ad0d8639da933ef07c36d0305bba94c84c727e2aeb325a230765a3645ccbfc29: Status 404 returned error can't find the container with id ad0d8639da933ef07c36d0305bba94c84c727e2aeb325a230765a3645ccbfc29 Jul 23 19:00:57.460651 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:00:57.460619 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d1b014_17b7_4ec0_9af9_aa8b5beda1be.slice/crio-8ff1732ef7103385922bde5d8316853e055a96c312d611441d65f17c3ad76745 WatchSource:0}: Error finding container 8ff1732ef7103385922bde5d8316853e055a96c312d611441d65f17c3ad76745: Status 404 returned error can't find the container with id 8ff1732ef7103385922bde5d8316853e055a96c312d611441d65f17c3ad76745 Jul 23 19:00:57.825443 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.825356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:57.825443 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.825435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:57.825675 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.825547 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:57.825675 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.825580 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:57.825675 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.825630 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.825608427 +0000 UTC m=+7.403925176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:57.825675 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.825652 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.825640885 +0000 UTC m=+7.403957625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:57.927024 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.926746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:57.927024 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.926974 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:00:57.927024 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.926999 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:00:57.927024 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.927012 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:57.927354 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:57.927069 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:00:59.927051452 +0000 UTC m=+7.505368197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:57.970469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.970401 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 18:55:56 +0000 UTC" deadline="2028-02-20 14:52:07.648837167 +0000 UTC" Jul 23 19:00:57.970469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:57.970462 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13843h51m9.678378937s" Jul 23 19:00:58.041966 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.041935 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:58.042164 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:58.042124 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:00:58.042561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.042534 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:58.042714 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:58.042690 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:00:58.042780 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.042771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:58.042872 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:58.042839 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:00:58.059579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.059529 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4xq9v" event={"ID":"f9582889-69da-4fc1-9340-3998fd371a39","Type":"ContainerStarted","Data":"ad0d8639da933ef07c36d0305bba94c84c727e2aeb325a230765a3645ccbfc29"} Jul 23 19:00:58.064620 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.064589 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" event={"ID":"3f5e67c885e56ac5079a5f283c430044","Type":"ContainerStarted","Data":"6084ee056209f0966bcf6e24a6ef36847fca17f91e89beefcc255299a24d144d"} Jul 23 19:00:58.074598 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.074569 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" event={"ID":"0e92e899-d841-48d9-aefd-9492aa8784a0","Type":"ContainerStarted","Data":"ba4210ae5ce92293ef904836cf83586ddfa9e14f5e9d6029a290b7f53d9d6480"} Jul 23 19:00:58.080825 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.080761 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lwp75" event={"ID":"a0c70287-dd25-4cd7-b656-93a14d7605ef","Type":"ContainerStarted","Data":"a3dcc3a3e6748a56584ec4a0e0c8179bcd674540fbb82a4f583f971e68c49b3f"} Jul 23 19:00:58.084583 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.084558 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" event={"ID":"922464fd-cb2e-4510-b706-39615e4216bf","Type":"ContainerStarted","Data":"029cfaba6304adbab9bb2e498536bc32514290aa44e796b39243c44219a2986c"} Jul 23 19:00:58.089436 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.089355 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjv4k" event={"ID":"73d1b014-17b7-4ec0-9af9-aa8b5beda1be","Type":"ContainerStarted","Data":"8ff1732ef7103385922bde5d8316853e055a96c312d611441d65f17c3ad76745"} Jul 23 19:00:58.093906 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.093853 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ct8h8" event={"ID":"a956c0a0-2420-4040-913e-d714b4dc5418","Type":"ContainerStarted","Data":"96ded8e238693577b5c2d8d77ff10c74546e8d145bf25654a48fc2a859deefa2"} Jul 23 19:00:58.096512 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.096485 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" event={"ID":"a63dd2beb5b4bbbe932b9b8b6353efdf","Type":"ContainerStarted","Data":"fad3310c67be3dd818b4ee24a004ad86a253c0be7dd7dc780a285d2f6176c751"} Jul 23 19:00:58.100169 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.100107 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerStarted","Data":"8936e31457ac40ffd9cf5fd421c58e9b18c768b3cbd427076a25f289493d3337"} Jul 23 19:00:58.102093 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.102066 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6s2m7" event={"ID":"5bb7b0bd-5164-4341-8948-08dd57301b5b","Type":"ContainerStarted","Data":"0bd09564a3225df52cd51f4c75d6ccdced4cd5b590002f294e0901b17fe3998c"} Jul 23 19:00:58.105847 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:58.105803 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"e4dd38e8cc1fe856a48145cc896f579ee44586d19648b858b7f03eed0ab542fb"} Jul 23 19:00:59.843674 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:59.843633 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:00:59.844208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:59.843723 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:00:59.844208 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.843882 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:59.844208 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.843953 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:03.843929869 +0000 UTC m=+11.422246619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:00:59.844406 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.844393 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:59.844477 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.844449 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:03.844431994 +0000 UTC m=+11.422748745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:00:59.944901 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:00:59.944844 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:00:59.945112 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.945042 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:00:59.945112 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.945065 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:00:59.945112 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.945081 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:00:59.945302 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:00:59.945146 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:03.94512541 +0000 UTC m=+11.523442163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:00.041143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:00.041103 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:00.041143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:00.041141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:00.041399 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:00.041109 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:00.041399 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:00.041254 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:00.041399 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:00.041365 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:00.041557 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:00.041448 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:02.041277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:02.041242 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:02.041724 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:02.041369 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:02.041855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:02.041813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:02.041974 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:02.041952 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:02.042051 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:02.042022 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:02.042106 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:02.042094 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:03.878641 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:03.878597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:03.879110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:03.878665 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:03.879110 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.878803 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:03.879110 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.878815 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:03.879110 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.878880 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:11.878861181 +0000 UTC m=+19.457177929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:03.879110 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.878898 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:11.878888688 +0000 UTC m=+19.457205436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:03.979481 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:03.979213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:03.979481 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.979367 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:01:03.979481 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.979388 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:01:03.979481 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.979402 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:03.979481 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:03.979473 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:11.979453157 +0000 UTC m=+19.557769909 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:04.041367 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:04.041495 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:04.041616 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:04.041731 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:04.041765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:04.041989 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:04.041853 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:06.041545 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:06.041683 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:06.041765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:06.041866 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:06.041918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:06.042294 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:06.041982 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:08.041179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:08.041142 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:08.041179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:08.041175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:08.041679 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:08.041178 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:08.041679 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:08.041257 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:08.041679 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:08.041375 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:08.041679 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:08.041452 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:10.041690 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:10.041659 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:10.041690 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:10.041696 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:10.042238 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:10.041702 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:10.042238 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:10.041781 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:10.042238 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:10.041938 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:10.042238 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:10.042034 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:11.943408 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:11.943368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:11.943877 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:11.943437 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:11.943953 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:11.943871 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:11.943953 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:11.943908 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:11.944056 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:11.943968 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:27.943944169 +0000 UTC m=+35.522260917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:11.944056 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:11.943991 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:27.943978786 +0000 UTC m=+35.522295521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:12.042177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:12.042135 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:12.042347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:12.042158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:12.042347 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.042279 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:12.042347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:12.042293 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:12.042507 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.042390 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:12.042557 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.042528 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:12.044287 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:12.044258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:12.044472 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.044452 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:01:12.044539 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.044483 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:01:12.044539 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.044496 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:12.044623 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:12.044559 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:28.044539995 +0000 UTC m=+35.622856730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:14.041746 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:14.041714 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:14.042176 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:14.041712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:14.042176 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:14.041862 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:14.042176 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:14.041916 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:14.042176 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:14.041972 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:14.042176 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:14.042074 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:16.041529 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:16.041491 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:16.042029 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:16.041505 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:16.042029 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:16.041624 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:16.042029 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:16.041733 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:16.042029 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:16.042017 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:16.042220 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:16.042115 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:18.041617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.041421 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:18.042100 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:18.041752 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:18.042100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.041820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:18.042100 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:18.041942 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:18.042100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.041988 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:18.042100 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:18.042080 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:18.164974 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.164872 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" event={"ID":"a63dd2beb5b4bbbe932b9b8b6353efdf","Type":"ContainerStarted","Data":"6261b21884ec7a6e5279131da3ba88516c38cb964bff4759b1b471796069c3e5"} Jul 23 19:01:18.168715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.167967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6s2m7" event={"ID":"5bb7b0bd-5164-4341-8948-08dd57301b5b","Type":"ContainerStarted","Data":"0f5652426d5b236559a0518358b83e57b353705aa07e914bd1efec732e976a3c"} Jul 23 19:01:18.169982 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.169915 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"d42396cb682f1bd6558b845a44ea251dad51ae8876a46d7d6b85d6d2943ce16f"} Jul 23 19:01:18.172296 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.172031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" event={"ID":"922464fd-cb2e-4510-b706-39615e4216bf","Type":"ContainerStarted","Data":"a6c9ffc49397008705147365b8dd5caba82666f01279fcbe9bc6c68d58c235a9"} Jul 23 19:01:18.183504 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.182267 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-123.ec2.internal" podStartSLOduration=22.182249507 podStartE2EDuration="22.182249507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.182216914 +0000 UTC m=+25.760533670" watchObservedRunningTime="2026-07-23 19:01:18.182249507 +0000 UTC m=+25.760566263" Jul 23 19:01:18.224102 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.224050 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gqpfc" podStartSLOduration=2.6295371210000003 podStartE2EDuration="23.224035074s" totalImagesPullingTime="20.594497953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.22376498 +0000 UTC m=+25.802081754" watchObservedRunningTime="2026-07-23 19:01:18.224035074 +0000 UTC m=+25.802351829" Jul 23 19:01:18.224317 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:18.224295 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6s2m7" podStartSLOduration=2.454869317 podStartE2EDuration="23.224289735s" totalImagesPullingTime="20.769420418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:18.202602732 +0000 UTC m=+25.780919486" watchObservedRunningTime="2026-07-23 19:01:18.224289735 +0000 UTC m=+25.802606490" Jul 23 19:01:19.175594 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.175358 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjv4k" event={"ID":"73d1b014-17b7-4ec0-9af9-aa8b5beda1be","Type":"ContainerStarted","Data":"48b8a6de1d3629d84cb12fb38af117438c4d4278fd265864737d6d4d522832ff"} Jul 23 19:01:19.177070 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.177040 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ct8h8" event={"ID":"a956c0a0-2420-4040-913e-d714b4dc5418","Type":"ContainerStarted","Data":"5308c4f86d36b9df349ccac841c81f0ab6100651d53f3fef29a71010dea4c9fc"} Jul 23 19:01:19.178724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.178699 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="31e19a1ef0ab072c97eb34e722f088a19d847436f1f38cd4de08561f32ebdd34" exitCode=0 Jul 23 19:01:19.178821 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.178780 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"31e19a1ef0ab072c97eb34e722f088a19d847436f1f38cd4de08561f32ebdd34"} Jul 23 19:01:19.183715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.183693 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"2eb80c50906137d8f9290249017852011d98266f9f7a836d8df65d8e60eab6f6"} Jul 23 19:01:19.183809 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.183767 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"56e9bd525e7b19059fc11f58fda28e885aea336ee370483c63e51b8d463d292a"} Jul 23 19:01:19.183896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.183820 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"a57fd5eaffff955960b5cae39ad1df358e2f7ba2f7b74da7787f070761a25468"} Jul 23 19:01:19.183896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.183881 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"ac3568abec4331f31fae95e7f05b2ddfc9fcaa1b7a0de30eb1d563ad24f2a6c7"} Jul 23 19:01:19.183986 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.183925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"1c56dfccfb9fab0b05f3a8011de1809621198dbbcf0b3f5662ec54d77a3d80d9"} Jul 23 19:01:19.185257 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.185228 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4xq9v" event={"ID":"f9582889-69da-4fc1-9340-3998fd371a39","Type":"ContainerStarted","Data":"ba9a66380ec18581bfed202f59479c516ccb9c13f3e1f4674b3edc577af6942a"} Jul 23 19:01:19.186884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.186859 2576 generic.go:356] "Generic (PLEG): container finished" podID="3f5e67c885e56ac5079a5f283c430044" containerID="76ee257ee1ab075ddc8f0d1de3f4a80350a605a4970a926bfed384204d8119a3" exitCode=0 Jul 23 19:01:19.186983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.186949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" event={"ID":"3f5e67c885e56ac5079a5f283c430044","Type":"ContainerDied","Data":"76ee257ee1ab075ddc8f0d1de3f4a80350a605a4970a926bfed384204d8119a3"} Jul 23 19:01:19.188511 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.188493 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" event={"ID":"0e92e899-d841-48d9-aefd-9492aa8784a0","Type":"ContainerStarted","Data":"541c2dc4b7f13ce86406bb05c9a3115e793938b35d9c4d110ea9db2d8f581cd2"} Jul 23 19:01:19.189939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.189917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lwp75" event={"ID":"a0c70287-dd25-4cd7-b656-93a14d7605ef","Type":"ContainerStarted","Data":"1df33a6736a8858afb6ff0f9862e87c38e11e9ad2a9c12de0568fd7d8102aee4"} Jul 23 19:01:19.192947 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.192910 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ct8h8" podStartSLOduration=3.687561922 podStartE2EDuration="24.192896265s" totalImagesPullingTime="20.505334343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:19.192757659 +0000 UTC m=+26.771074416" watchObservedRunningTime="2026-07-23 19:01:19.192896265 +0000 UTC m=+26.771213022" Jul 23 19:01:19.247141 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.247090 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4xq9v" podStartSLOduration=3.8642281609999998 podStartE2EDuration="24.247072714s" totalImagesPullingTime="20.382844553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:19.247012994 +0000 UTC m=+26.825329749" watchObservedRunningTime="2026-07-23 19:01:19.247072714 +0000 UTC m=+26.825389471" Jul 23 19:01:19.270959 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.269508 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lwp75" podStartSLOduration=2.699405893 podStartE2EDuration="23.269482969s" totalImagesPullingTime="20.570077076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:19.268705907 +0000 UTC m=+26.847022663" watchObservedRunningTime="2026-07-23 19:01:19.269482969 +0000 UTC m=+26.847799724" Jul 23 19:01:19.642017 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:19.641984 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 19:01:20.042172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.042140 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:20.042172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.042170 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:20.042524 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.042174 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:20.042524 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:20.042263 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:20.042524 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:20.042373 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:20.042524 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:20.042450 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:20.194687 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.194628 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" event={"ID":"3f5e67c885e56ac5079a5f283c430044","Type":"ContainerStarted","Data":"75949c104014eb5d37c03be324201fdd85ae450283be64c8de911687f2a4b4bc"} Jul 23 19:01:20.196930 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.196895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" event={"ID":"0e92e899-d841-48d9-aefd-9492aa8784a0","Type":"ContainerStarted","Data":"3348c81598d1ec114dacf6bc71b05a255a9a1f878bd1e9e01e0ad40e2c42d891"} Jul 23 19:01:20.211978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.211923 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-123.ec2.internal" podStartSLOduration=24.211907023 podStartE2EDuration="24.211907023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:20.211225248 +0000 UTC m=+27.789542005" watchObservedRunningTime="2026-07-23 19:01:20.211907023 +0000 UTC m=+27.790223778" Jul 23 19:01:20.228001 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.227900 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T19:01:19.642003454Z","UUID":"c23f47e1-a6d6-467f-b880-2a930b4fb2ca","Handler":null,"Name":"","Endpoint":""} Jul 23 19:01:20.228821 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.228777 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kjv4k" podStartSLOduration=4.871981427 podStartE2EDuration="25.228759236s" totalImagesPullingTime="20.356777809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:20.228581653 +0000 UTC m=+27.806898409" watchObservedRunningTime="2026-07-23 19:01:20.228759236 +0000 UTC m=+27.807075992" Jul 23 19:01:20.229555 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.229534 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 19:01:20.229635 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:20.229575 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 19:01:21.203505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:21.203471 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"a7061818fc31176f8c1947c29e12e05f517bf68365a6bfa212c0538122286e72"} Jul 23 19:01:21.680754 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:21.680729 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:01:21.684349 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:21.684152 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:01:22.041450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.041418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:22.041450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.041423 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:22.041692 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.041434 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:22.041692 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:22.041566 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:22.041692 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:22.041677 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:22.041872 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:22.041797 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:22.208799 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.208758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" event={"ID":"0e92e899-d841-48d9-aefd-9492aa8784a0","Type":"ContainerStarted","Data":"a197c49900d9efdcfe5d08f2052ac289caa6a73e002df99df68d659829bff70a"} Jul 23 19:01:22.209306 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.208990 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:01:22.209643 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.209627 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lwp75" Jul 23 19:01:22.237303 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:22.237245 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ssfj" podStartSLOduration=2.98935059 podStartE2EDuration="26.237228625s" totalImagesPullingTime="23.247878035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:22.237090687 +0000 UTC m=+29.815407443" watchObservedRunningTime="2026-07-23 19:01:22.237228625 +0000 UTC m=+29.815545380" Jul 23 19:01:24.041715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.041518 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:24.042336 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.041551 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:24.042336 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:24.041801 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:24.042336 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:24.041894 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:24.042336 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.041555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:24.042336 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:24.042013 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:24.215263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.215227 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="4932e123b744e9bcbd92013d46fe63a9f55fe57038364e5cf37b0b37052f98fd" exitCode=0 Jul 23 19:01:24.215495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.215313 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"4932e123b744e9bcbd92013d46fe63a9f55fe57038364e5cf37b0b37052f98fd"} Jul 23 19:01:24.219134 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.219104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" event={"ID":"3f45f70b-fce7-4786-9a4b-9661f273fa8e","Type":"ContainerStarted","Data":"4c40e0c6df5cd5728cf9ccefd2d1d77acf5886b7ab66f452b9b9b27668b03276"} Jul 23 19:01:24.219423 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.219399 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:24.219522 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.219473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:24.234537 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.234507 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:24.282987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:24.282931 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" podStartSLOduration=8.645952903 podStartE2EDuration="29.282916189s" totalImagesPullingTime="20.636963286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:24.282209378 +0000 UTC m=+31.860526133" watchObservedRunningTime="2026-07-23 19:01:24.282916189 +0000 UTC m=+31.861232943" Jul 23 19:01:25.221157 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.221128 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:25.237332 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.237302 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:25.849275 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.849072 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2q6nq"] Jul 23 19:01:25.849440 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.849426 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:25.849554 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:25.849535 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:25.849819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.849796 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvmj8"] Jul 23 19:01:25.849940 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.849913 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:25.850010 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:25.849991 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:25.850584 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.850562 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j6lsv"] Jul 23 19:01:25.850677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:25.850648 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:25.850738 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:25.850717 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:26.224652 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:26.224571 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="ae5d385224257770fdce9871a1f289fb118615fd0d1a0fc460d63e64a0486ebc" exitCode=0 Jul 23 19:01:26.225043 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:26.224655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"ae5d385224257770fdce9871a1f289fb118615fd0d1a0fc460d63e64a0486ebc"} Jul 23 19:01:27.042202 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:27.042171 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:27.042394 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.042317 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:27.042394 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:27.042365 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:27.042517 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.042484 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:27.957179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:27.957142 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:27.957610 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:27.957210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:27.957610 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.957286 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:27.957610 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.957322 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:27.957610 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.957350 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs podName:68a4014d-9b37-458d-b82d-4a29ef017488 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:59.957335557 +0000 UTC m=+67.535652290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs") pod "network-metrics-daemon-kvmj8" (UID: "68a4014d-9b37-458d-b82d-4a29ef017488") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 19:01:27.957610 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:27.957373 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret podName:8c29a651-9e48-4ff9-842c-ccd602552695 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:59.957356403 +0000 UTC m=+67.535673138 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret") pod "global-pull-secret-syncer-2q6nq" (UID: "8c29a651-9e48-4ff9-842c-ccd602552695") : object "kube-system"/"original-pull-secret" not registered Jul 23 19:01:28.041902 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:28.041874 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:28.042062 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:28.041968 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:28.058111 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:28.058085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:28.058235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:28.058223 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 19:01:28.058278 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:28.058241 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 19:01:28.058278 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:28.058249 2576 projected.go:196] Error preparing data for projected volume kube-api-access-m6cwq for pod openshift-network-diagnostics/network-check-target-j6lsv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:28.058336 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:28.058293 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq podName:e28b1f13-a8c3-469d-a2fd-1348bc3645c3 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:00.058279497 +0000 UTC m=+67.636596230 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m6cwq" (UniqueName: "kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq") pod "network-check-target-j6lsv" (UID: "e28b1f13-a8c3-469d-a2fd-1348bc3645c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 19:01:28.231649 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:28.231613 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="39435c6018ff42609c1a07bbf390e5a795d5d992b2a388c13d7b28e154667d9c" exitCode=0 Jul 23 19:01:28.231827 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:28.231674 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"39435c6018ff42609c1a07bbf390e5a795d5d992b2a388c13d7b28e154667d9c"} Jul 23 19:01:29.042121 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:29.042087 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:29.042583 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:29.042233 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kvmj8" podUID="68a4014d-9b37-458d-b82d-4a29ef017488" Jul 23 19:01:29.042583 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:29.042281 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:29.042583 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:29.042392 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j6lsv" podUID="e28b1f13-a8c3-469d-a2fd-1348bc3645c3" Jul 23 19:01:30.042021 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.041989 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:30.042218 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:30.042118 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2q6nq" podUID="8c29a651-9e48-4ff9-842c-ccd602552695" Jul 23 19:01:30.778952 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.778915 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-123.ec2.internal" event="NodeReady" Jul 23 19:01:30.779134 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.779060 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 19:01:30.832818 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.832787 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:01:30.836624 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.836599 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.838951 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.838921 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-wmcln"] Jul 23 19:01:30.841880 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.841852 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td"] Jul 23 19:01:30.841993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.841945 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.841993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.841983 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 19:01:30.842351 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.842310 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 19:01:30.842582 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.842530 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 19:01:30.845412 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.845394 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv"] Jul 23 19:01:30.845574 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.845549 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:30.847074 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:30.846669 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 23 19:01:30.847074 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:30.846704 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-insights-serving-cert\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" Jul 23 19:01:30.848035 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:30.848010 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 23 19:01:30.848116 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:30.848011 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" Jul 23 19:01:30.848231 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.848217 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg"] Jul 23 19:01:30.848345 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.848328 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:30.849656 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.849638 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6zflj\"" Jul 23 19:01:30.850173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.849890 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4m5jd\"" Jul 23 19:01:30.850173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.849921 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 19:01:30.850173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.849976 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.850173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.850003 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 19:01:30.850401 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.850371 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nv7r4\"" Jul 23 19:01:30.851592 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.851575 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv"] Jul 23 19:01:30.851711 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.851695 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:30.852625 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.852606 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.852805 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.852674 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 19:01:30.853219 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.852998 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.853482 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.853460 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.853579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.853510 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gnb95\"" Jul 23 19:01:30.854412 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.854393 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-29gmr"] Jul 23 19:01:30.854516 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.854501 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" Jul 23 19:01:30.859256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.856296 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 19:01:30.859256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.858164 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 19:01:30.859256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.858176 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.859256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.858322 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.859256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.859045 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c"] Jul 23 19:01:30.859541 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.859377 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:30.862541 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.862350 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9c889d5d-pjpft"] Jul 23 19:01:30.863089 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.863069 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:30.863590 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.863573 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-h2gsg\"" Jul 23 19:01:30.864188 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.863963 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.865069 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.865052 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:01:30.865198 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.865122 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.865811 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.865697 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-22sjl\"" Jul 23 19:01:30.865959 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.865831 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.866903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.866599 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.866903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.866617 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.866903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.866637 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x8pp6\"" Jul 23 19:01:30.867109 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.867016 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 19:01:30.867468 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.867446 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 19:01:30.868019 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.867729 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.872343 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872322 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-29gmr"] Jul 23 19:01:30.872533 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872515 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 19:01:30.872617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872538 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-w5v8m\"" Jul 23 19:01:30.872617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872558 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.872617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872580 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 19:01:30.872770 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.872522 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 19:01:30.873066 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.873046 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 19:01:30.875064 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.875048 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 19:01:30.877079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.877059 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.877171 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.877150 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.877298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.877276 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 19:01:30.878062 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.878043 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n2skf\"" Jul 23 19:01:30.879675 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.879616 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 19:01:30.890449 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.890418 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 19:01:30.893416 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.893391 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td"] Jul 23 19:01:30.894819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.894779 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c"] Jul 23 19:01:30.895477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.895456 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv"] Jul 23 19:01:30.896143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.896123 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 19:01:30.896354 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.896339 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg"] Jul 23 19:01:30.897132 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.897113 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-wmcln"] Jul 23 19:01:30.903201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.903183 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9c889d5d-pjpft"] Jul 23 19:01:30.904683 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.904660 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-54kmk"] Jul 23 19:01:30.912000 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.911978 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rsv86"] Jul 23 19:01:30.912137 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.912122 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:30.915122 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.914797 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv"] Jul 23 19:01:30.915122 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.914951 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 19:01:30.915122 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.914974 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 19:01:30.915315 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.915265 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:30.915374 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.915324 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kngdw\"" Jul 23 19:01:30.916295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.916077 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 19:01:30.917862 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.917828 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-54kmk"] Jul 23 19:01:30.918369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.918352 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 19:01:30.918451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.918417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fgf46\"" Jul 23 19:01:30.918681 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.918662 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 19:01:30.925375 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.925280 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rsv86"] Jul 23 19:01:30.978503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978467 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.978503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4dd8699-497d-4f9e-988f-f688f3aafd46-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8dddf0-949c-4228-a169-88a50e857fb2-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggxf\" (UniqueName: \"kubernetes.io/projected/5bfefa19-7c7e-4f52-a53e-a473d73137a5-kube-api-access-4ggxf\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkv5j\" (UniqueName: \"kubernetes.io/projected/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-kube-api-access-gkv5j\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.978736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978737 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-config\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-trusted-ca\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cee55273-2d39-4ff5-81dc-534545313734-serving-cert\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978947 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxwn7\" (UniqueName: \"kubernetes.io/projected/6c272505-3b03-497e-ba84-38743658d5b2-kube-api-access-rxwn7\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.978982 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjphb\" (UniqueName: \"kubernetes.io/projected/e4dd8699-497d-4f9e-988f-f688f3aafd46-kube-api-access-pjphb\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979078 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-tmp\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979158 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5bg9\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fr9n\" (UniqueName: \"kubernetes.io/projected/57d361da-ffa5-491c-8fa8-725e8a97513b-kube-api-access-8fr9n\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979214 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-default-certificate\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-config\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:30.979338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr8l4\" (UniqueName: \"kubernetes.io/projected/8c8dddf0-949c-4228-a169-88a50e857fb2-kube-api-access-qr8l4\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr8cx\" (UniqueName: \"kubernetes.io/projected/e6a1a6d3-7ed0-48ad-a745-fa49b2aef942-kube-api-access-pr8cx\") pod \"volume-data-source-validator-78cf4c6f6-qbdwv\" (UID: \"e6a1a6d3-7ed0-48ad-a745-fa49b2aef942\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnppj\" (UniqueName: \"kubernetes.io/projected/cee55273-2d39-4ff5-81dc-534545313734-kube-api-access-dnppj\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8dddf0-949c-4228-a169-88a50e857fb2-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979577 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979594 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-snapshots\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979609 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-stats-auth\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.979631 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979622 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.979977 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:30.979977 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:30.979673 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57d361da-ffa5-491c-8fa8-725e8a97513b-serving-cert\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.041663 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.041595 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:01:31.041802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.041593 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:01:31.048225 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.048203 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 19:01:31.048638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.048203 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 19:01:31.048638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.048267 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8qs8\"" Jul 23 19:01:31.048638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.048522 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 19:01:31.048638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.048598 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5ztj\"" Jul 23 19:01:31.080828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080798 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080883 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-trusted-ca\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080903 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cee55273-2d39-4ff5-81dc-534545313734-serving-cert\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080922 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxwn7\" (UniqueName: \"kubernetes.io/projected/6c272505-3b03-497e-ba84-38743658d5b2-kube-api-access-rxwn7\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080943 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjphb\" (UniqueName: \"kubernetes.io/projected/e4dd8699-497d-4f9e-988f-f688f3aafd46-kube-api-access-pjphb\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.081008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.080992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.081327 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.081327 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081227 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-tmp\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.081327 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081261 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5bg9\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.081327 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fr9n\" (UniqueName: \"kubernetes.io/projected/57d361da-ffa5-491c-8fa8-725e8a97513b-kube-api-access-8fr9n\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.081586 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69e66d5f-e0db-426b-a333-a7f9243f1745-config-volume\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.081653 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081618 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-tmp\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.081715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-default-certificate\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.081760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-config\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.081760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081750 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr8l4\" (UniqueName: \"kubernetes.io/projected/8c8dddf0-949c-4228-a169-88a50e857fb2-kube-api-access-qr8l4\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.081871 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081779 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.081871 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081804 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e461165-4a68-4b19-9e61-10616c8eed5d-cert\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.081955 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e66d5f-e0db-426b-a333-a7f9243f1745-tmp-dir\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.081997 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081963 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.081997 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.081988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.082084 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082012 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr8cx\" (UniqueName: \"kubernetes.io/projected/e6a1a6d3-7ed0-48ad-a745-fa49b2aef942-kube-api-access-pr8cx\") pod \"volume-data-source-validator-78cf4c6f6-qbdwv\" (UID: \"e6a1a6d3-7ed0-48ad-a745-fa49b2aef942\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" Jul 23 19:01:31.082084 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-trusted-ca\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.082084 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnppj\" (UniqueName: \"kubernetes.io/projected/cee55273-2d39-4ff5-81dc-534545313734-kube-api-access-dnppj\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.082084 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082065 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8dddf0-949c-4228-a169-88a50e857fb2-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.082281 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.082091 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:31.082281 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.082160 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.582141271 +0000 UTC m=+39.160458018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:31.082281 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082199 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.082405 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082092 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.082457 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082422 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-snapshots\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.082500 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082463 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-stats-auth\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.082500 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.082609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082511 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.082609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082533 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57d361da-ffa5-491c-8fa8-725e8a97513b-serving-cert\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.082609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082560 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.082609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4dd8699-497d-4f9e-988f-f688f3aafd46-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.082609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082604 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8dddf0-949c-4228-a169-88a50e857fb2-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt6zc\" (UniqueName: \"kubernetes.io/projected/8e461165-4a68-4b19-9e61-10616c8eed5d-kube-api-access-dt6zc\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082683 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggxf\" (UniqueName: \"kubernetes.io/projected/5bfefa19-7c7e-4f52-a53e-a473d73137a5-kube-api-access-4ggxf\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082722 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082748 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxxck\" (UniqueName: \"kubernetes.io/projected/69e66d5f-e0db-426b-a333-a7f9243f1745-kube-api-access-mxxck\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkv5j\" (UniqueName: \"kubernetes.io/projected/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-kube-api-access-gkv5j\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.082808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082803 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/57d361da-ffa5-491c-8fa8-725e8a97513b-config\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.083219 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.082826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-config\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.083308 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083262 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.083364 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083350 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c8dddf0-949c-4228-a169-88a50e857fb2-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083447 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083452 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-config\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083502 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls podName:6c272505-3b03-497e-ba84-38743658d5b2 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.583485016 +0000 UTC m=+39.161801766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-46qdg" (UID: "6c272505-3b03-497e-ba84-38743658d5b2") : secret "samples-operator-tls" not found Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083525 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083552 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083636 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083700 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.583684054 +0000 UTC m=+39.162000800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083892 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.583874289 +0000 UTC m=+39.162191038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.083953 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cee55273-2d39-4ff5-81dc-534545313734-snapshots\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083980 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:31.083994 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.083992 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:31.084621 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.084041 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.584023156 +0000 UTC m=+39.162339889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:31.084621 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.084418 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e4dd8699-497d-4f9e-988f-f688f3aafd46-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.086635 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.086609 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.086635 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.086627 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.086635 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.086637 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-stats-auth\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.086928 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.086883 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-serving-cert\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.087178 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.087155 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-default-certificate\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.087769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.087745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/57d361da-ffa5-491c-8fa8-725e8a97513b-serving-cert\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.088298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.088279 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c8dddf0-949c-4228-a169-88a50e857fb2-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.109692 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.109625 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjphb\" (UniqueName: \"kubernetes.io/projected/e4dd8699-497d-4f9e-988f-f688f3aafd46-kube-api-access-pjphb\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.109692 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.109677 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxwn7\" (UniqueName: \"kubernetes.io/projected/6c272505-3b03-497e-ba84-38743658d5b2-kube-api-access-rxwn7\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:31.110137 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.110115 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkv5j\" (UniqueName: \"kubernetes.io/projected/aa7a3cf6-e8be-4448-a8b5-14524115c6b4-kube-api-access-gkv5j\") pod \"service-ca-operator-5bcbc94c78-wj64c\" (UID: \"aa7a3cf6-e8be-4448-a8b5-14524115c6b4\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.110675 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.110630 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fr9n\" (UniqueName: \"kubernetes.io/projected/57d361da-ffa5-491c-8fa8-725e8a97513b-kube-api-access-8fr9n\") pod \"console-operator-565b7b569d-29gmr\" (UID: \"57d361da-ffa5-491c-8fa8-725e8a97513b\") " pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.110919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.110877 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.111012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.110984 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr8l4\" (UniqueName: \"kubernetes.io/projected/8c8dddf0-949c-4228-a169-88a50e857fb2-kube-api-access-qr8l4\") pod \"kube-storage-version-migrator-operator-7658d9b749-ks8td\" (UID: \"8c8dddf0-949c-4228-a169-88a50e857fb2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.111116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.111097 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5bg9\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.111947 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.111918 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr8cx\" (UniqueName: \"kubernetes.io/projected/e6a1a6d3-7ed0-48ad-a745-fa49b2aef942-kube-api-access-pr8cx\") pod \"volume-data-source-validator-78cf4c6f6-qbdwv\" (UID: \"e6a1a6d3-7ed0-48ad-a745-fa49b2aef942\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" Jul 23 19:01:31.112778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.112760 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggxf\" (UniqueName: \"kubernetes.io/projected/5bfefa19-7c7e-4f52-a53e-a473d73137a5-kube-api-access-4ggxf\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.149383 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.149227 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:31.167437 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.167404 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" Jul 23 19:01:31.183978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.183946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69e66d5f-e0db-426b-a333-a7f9243f1745-config-volume\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.184107 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184025 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e461165-4a68-4b19-9e61-10616c8eed5d-cert\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.184107 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184046 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e66d5f-e0db-426b-a333-a7f9243f1745-tmp-dir\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.184279 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184256 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt6zc\" (UniqueName: \"kubernetes.io/projected/8e461165-4a68-4b19-9e61-10616c8eed5d-kube-api-access-dt6zc\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.184360 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxxck\" (UniqueName: \"kubernetes.io/projected/69e66d5f-e0db-426b-a333-a7f9243f1745-kube-api-access-mxxck\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.184360 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.184452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184380 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69e66d5f-e0db-426b-a333-a7f9243f1745-tmp-dir\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.184503 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.184476 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:31.184556 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.184530 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:31.684512837 +0000 UTC m=+39.262829593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:31.184556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.184534 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/69e66d5f-e0db-426b-a333-a7f9243f1745-config-volume\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.187112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.187091 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e461165-4a68-4b19-9e61-10616c8eed5d-cert\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.191415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.191386 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" Jul 23 19:01:31.195491 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.195468 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxxck\" (UniqueName: \"kubernetes.io/projected/69e66d5f-e0db-426b-a333-a7f9243f1745-kube-api-access-mxxck\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.195679 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.195660 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt6zc\" (UniqueName: \"kubernetes.io/projected/8e461165-4a68-4b19-9e61-10616c8eed5d-kube-api-access-dt6zc\") pod \"ingress-canary-54kmk\" (UID: \"8e461165-4a68-4b19-9e61-10616c8eed5d\") " pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.200645 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.200625 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:31.209173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.209153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" Jul 23 19:01:31.226436 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.226408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-54kmk" Jul 23 19:01:31.248017 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.247984 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:31.588192 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.588151 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.588206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.588228 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.588258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.588285 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588320 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588324 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:31.588382 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588384 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.588365849 +0000 UTC m=+40.166682601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588403 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.588392994 +0000 UTC m=+40.166709731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588409 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588420 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.588412334 +0000 UTC m=+40.166729074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588422 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588427 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588464 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls podName:6c272505-3b03-497e-ba84-38743658d5b2 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.588456962 +0000 UTC m=+40.166773696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-46qdg" (UID: "6c272505-3b03-497e-ba84-38743658d5b2") : secret "samples-operator-tls" not found Jul 23 19:01:31.588732 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.588477 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.588472222 +0000 UTC m=+40.166788968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:31.689714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.689672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:31.689936 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.689817 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:31.689936 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:31.689908 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.689888203 +0000 UTC m=+40.268204952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:31.786181 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.786150 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 19:01:31.793975 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.793948 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cee55273-2d39-4ff5-81dc-534545313734-serving-cert\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:31.851100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:31.851021 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 19:01:32.041414 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.041380 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:01:32.044518 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.044497 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 19:01:32.084084 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.084056 2576 configmap.go:193] Couldn't get configMap openshift-insights/service-ca-bundle: failed to sync configmap cache: timed out waiting for the condition Jul 23 19:01:32.084458 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.084130 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle podName:cee55273-2d39-4ff5-81dc-534545313734 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:32.584109077 +0000 UTC m=+40.162425811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle") pod "insights-operator-555d9d7885-wmcln" (UID: "cee55273-2d39-4ff5-81dc-534545313734") : failed to sync configmap cache: timed out waiting for the condition Jul 23 19:01:32.281418 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.279395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 19:01:32.305196 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.305134 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnppj\" (UniqueName: \"kubernetes.io/projected/cee55273-2d39-4ff5-81dc-534545313734-kube-api-access-dnppj\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:32.398892 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.398520 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 19:01:32.421076 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.421047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv"] Jul 23 19:01:32.426234 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.426212 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c"] Jul 23 19:01:32.429744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.429715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-29gmr"] Jul 23 19:01:32.432437 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.432415 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-54kmk"] Jul 23 19:01:32.434939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.434917 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td"] Jul 23 19:01:32.438393 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.438364 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e461165_4a68_4b19_9e61_10616c8eed5d.slice/crio-fd2a8e1779dbd45fd8199dbc391c704b05659ea9d9d7e86b192e2bb32c31fc2f WatchSource:0}: Error finding container fd2a8e1779dbd45fd8199dbc391c704b05659ea9d9d7e86b192e2bb32c31fc2f: Status 404 returned error can't find the container with id fd2a8e1779dbd45fd8199dbc391c704b05659ea9d9d7e86b192e2bb32c31fc2f Jul 23 19:01:32.439007 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.438974 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d361da_ffa5_491c_8fa8_725e8a97513b.slice/crio-863baeade8f8d4fd2dc0991a9027ac67e9cac5ab8202f083fa5318864e7b92d2 WatchSource:0}: Error finding container 863baeade8f8d4fd2dc0991a9027ac67e9cac5ab8202f083fa5318864e7b92d2: Status 404 returned error can't find the container with id 863baeade8f8d4fd2dc0991a9027ac67e9cac5ab8202f083fa5318864e7b92d2 Jul 23 19:01:32.439580 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.439560 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a1a6d3_7ed0_48ad_a745_fa49b2aef942.slice/crio-84b2bf28ee0412920ca84c003237842d540fd0013c7e088c976796e7ba444a3c WatchSource:0}: Error finding container 84b2bf28ee0412920ca84c003237842d540fd0013c7e088c976796e7ba444a3c: Status 404 returned error can't find the container with id 84b2bf28ee0412920ca84c003237842d540fd0013c7e088c976796e7ba444a3c Jul 23 19:01:32.441194 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.441071 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8dddf0_949c_4228_a169_88a50e857fb2.slice/crio-7e499442cfa70e6fe7bf900bd21cc85e216c46b52a44094c1b3b6c027104b94c WatchSource:0}: Error finding container 7e499442cfa70e6fe7bf900bd21cc85e216c46b52a44094c1b3b6c027104b94c: Status 404 returned error can't find the container with id 7e499442cfa70e6fe7bf900bd21cc85e216c46b52a44094c1b3b6c027104b94c Jul 23 19:01:32.441950 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.441733 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7a3cf6_e8be_4448_a8b5_14524115c6b4.slice/crio-d9d26e1cbd74aa9a35893151343e9e8b57df344c961606c7e31a4ee9e45f40d1 WatchSource:0}: Error finding container d9d26e1cbd74aa9a35893151343e9e8b57df344c961606c7e31a4ee9e45f40d1: Status 404 returned error can't find the container with id d9d26e1cbd74aa9a35893151343e9e8b57df344c961606c7e31a4ee9e45f40d1 Jul 23 19:01:32.598718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:32.598718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598738 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598776 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.598824 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598842 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598939 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598948 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.598901633 +0000 UTC m=+42.177218389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:32.598967 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598960 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598948 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.598969 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.598959314 +0000 UTC m=+42.177276056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.599001 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.599018 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.59900407 +0000 UTC m=+42.177320809 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.599037 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls podName:6c272505-3b03-497e-ba84-38743658d5b2 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.599031222 +0000 UTC m=+42.177347955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-46qdg" (UID: "6c272505-3b03-497e-ba84-38743658d5b2") : secret "samples-operator-tls" not found Jul 23 19:01:32.599229 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.599049 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.59904223 +0000 UTC m=+42.177358963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:32.599430 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.599307 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cee55273-2d39-4ff5-81dc-534545313734-service-ca-bundle\") pod \"insights-operator-555d9d7885-wmcln\" (UID: \"cee55273-2d39-4ff5-81dc-534545313734\") " pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:32.659350 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.659319 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-wmcln" Jul 23 19:01:32.700221 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.700176 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:32.700388 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.700322 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:32.700430 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:32.700398 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:34.70037599 +0000 UTC m=+42.278692743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:32.781529 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:32.781499 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-wmcln"] Jul 23 19:01:32.784288 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:32.784266 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee55273_2d39_4ff5_81dc_534545313734.slice/crio-e7c5fb58c9c4232b332847e714aaa031fbc87988da38b5750cd27fc942e95ad7 WatchSource:0}: Error finding container e7c5fb58c9c4232b332847e714aaa031fbc87988da38b5750cd27fc942e95ad7: Status 404 returned error can't find the container with id e7c5fb58c9c4232b332847e714aaa031fbc87988da38b5750cd27fc942e95ad7 Jul 23 19:01:33.249656 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.249460 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="12f444ef835b3cd5abee70ed0e1bf7023bf95e3cf35bff7dc98305abeddd65c2" exitCode=0 Jul 23 19:01:33.249656 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.249560 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"12f444ef835b3cd5abee70ed0e1bf7023bf95e3cf35bff7dc98305abeddd65c2"} Jul 23 19:01:33.253760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.253426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" event={"ID":"aa7a3cf6-e8be-4448-a8b5-14524115c6b4","Type":"ContainerStarted","Data":"d9d26e1cbd74aa9a35893151343e9e8b57df344c961606c7e31a4ee9e45f40d1"} Jul 23 19:01:33.259321 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.259294 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-wmcln" event={"ID":"cee55273-2d39-4ff5-81dc-534545313734","Type":"ContainerStarted","Data":"e7c5fb58c9c4232b332847e714aaa031fbc87988da38b5750cd27fc942e95ad7"} Jul 23 19:01:33.261207 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.261180 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" event={"ID":"8c8dddf0-949c-4228-a169-88a50e857fb2","Type":"ContainerStarted","Data":"7e499442cfa70e6fe7bf900bd21cc85e216c46b52a44094c1b3b6c027104b94c"} Jul 23 19:01:33.263195 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.263142 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-54kmk" event={"ID":"8e461165-4a68-4b19-9e61-10616c8eed5d","Type":"ContainerStarted","Data":"fd2a8e1779dbd45fd8199dbc391c704b05659ea9d9d7e86b192e2bb32c31fc2f"} Jul 23 19:01:33.265261 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.265235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" event={"ID":"e6a1a6d3-7ed0-48ad-a745-fa49b2aef942","Type":"ContainerStarted","Data":"84b2bf28ee0412920ca84c003237842d540fd0013c7e088c976796e7ba444a3c"} Jul 23 19:01:33.266820 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:33.266792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" event={"ID":"57d361da-ffa5-491c-8fa8-725e8a97513b","Type":"ContainerStarted","Data":"863baeade8f8d4fd2dc0991a9027ac67e9cac5ab8202f083fa5318864e7b92d2"} Jul 23 19:01:34.272645 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.272563 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" event={"ID":"e6a1a6d3-7ed0-48ad-a745-fa49b2aef942","Type":"ContainerStarted","Data":"acf700d48ebca1b954af7de989fccf3dce008cc3b7c278f93577eae78a7ddac5"} Jul 23 19:01:34.279222 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.279188 2576 generic.go:356] "Generic (PLEG): container finished" podID="892756f1-263f-46af-b14f-a38baa295c80" containerID="ac9bb71e67813df95b81502bfa4582b57dc422a88716ded71b7f7da814f0b8b1" exitCode=0 Jul 23 19:01:34.279414 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.279258 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerDied","Data":"ac9bb71e67813df95b81502bfa4582b57dc422a88716ded71b7f7da814f0b8b1"} Jul 23 19:01:34.294821 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.293994 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qbdwv" podStartSLOduration=8.714732146 podStartE2EDuration="10.293972138s" totalImagesPullingTime="1.579239992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:34.292743118 +0000 UTC m=+41.871059874" watchObservedRunningTime="2026-07-23 19:01:34.293972138 +0000 UTC m=+41.872288894" Jul 23 19:01:34.397248 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.397206 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:34.618579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.618490 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:34.618579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.618535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:34.618579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.618562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.618589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618655 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618675 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.618655429 +0000 UTC m=+46.196972177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618779 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618814 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618874 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.618860858 +0000 UTC m=+46.197177594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618786 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 19:01:34.618914 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618917 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.618907249 +0000 UTC m=+46.197223987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:34.619275 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.618932 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls podName:6c272505-3b03-497e-ba84-38743658d5b2 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.618924339 +0000 UTC m=+46.197241079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-46qdg" (UID: "6c272505-3b03-497e-ba84-38743658d5b2") : secret "samples-operator-tls" not found Jul 23 19:01:34.619275 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.618993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:34.619275 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.619075 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:34.619275 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.619097 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.619090819 +0000 UTC m=+46.197407552 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:34.719688 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.719649 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:34.719871 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.719816 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:34.719939 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:34.719898 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:38.719878951 +0000 UTC m=+46.298195690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:34.910166 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:34.910088 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:36.290521 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:36.290487 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:36.373650 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:36.373609 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:36.465520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:36.465483 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:36.566213 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:36.566127 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:37.355737 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:37.355686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:37.425559 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:37.425527 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:38.293446 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.293412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44x8" event={"ID":"892756f1-263f-46af-b14f-a38baa295c80","Type":"ContainerStarted","Data":"94954116142c1a022fcfaaf591004e64a41eb282378e7e7b0ef1b5a148d5a0d2"} Jul 23 19:01:38.296137 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.296076 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" event={"ID":"aa7a3cf6-e8be-4448-a8b5-14524115c6b4","Type":"ContainerStarted","Data":"fad5ae0c90708f2783f189c9e392ffdb0b7f25806b7e038e53004e810f56df91"} Jul 23 19:01:38.297754 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.297729 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" event={"ID":"8c8dddf0-949c-4228-a169-88a50e857fb2","Type":"ContainerStarted","Data":"32c23383ceb43d151133c3086df5b6a9b63f0d66fcca130c73610e0243c2f5bb"} Jul 23 19:01:38.299456 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.299421 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-54kmk" event={"ID":"8e461165-4a68-4b19-9e61-10616c8eed5d","Type":"ContainerStarted","Data":"5432710f7e5f2b7bb851b40140ec5bba27cc123fe264eed45e95436d0ea67b23"} Jul 23 19:01:38.301877 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.301856 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/0.log" Jul 23 19:01:38.301998 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.301893 2576 generic.go:356] "Generic (PLEG): container finished" podID="57d361da-ffa5-491c-8fa8-725e8a97513b" containerID="bc3ec466fba64e8d63970973167cd996c4b009678a0108a3d11c3d90357f2417" exitCode=255 Jul 23 19:01:38.301998 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.301920 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" event={"ID":"57d361da-ffa5-491c-8fa8-725e8a97513b","Type":"ContainerDied","Data":"bc3ec466fba64e8d63970973167cd996c4b009678a0108a3d11c3d90357f2417"} Jul 23 19:01:38.302179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.302161 2576 scope.go:117] "RemoveContainer" containerID="bc3ec466fba64e8d63970973167cd996c4b009678a0108a3d11c3d90357f2417" Jul 23 19:01:38.325277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.325234 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p44x8" podStartSLOduration=10.721994568 podStartE2EDuration="42.325219812s" totalImagesPullingTime="31.603225244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:38.323892858 +0000 UTC m=+45.902209613" watchObservedRunningTime="2026-07-23 19:01:38.325219812 +0000 UTC m=+45.903536566" Jul 23 19:01:38.369380 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.369315 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" podStartSLOduration=5.27944835 podStartE2EDuration="10.369293144s" totalImagesPullingTime="5.089844794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:38.368197966 +0000 UTC m=+45.946514748" watchObservedRunningTime="2026-07-23 19:01:38.369293144 +0000 UTC m=+45.947609900" Jul 23 19:01:38.402260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.402204 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" podStartSLOduration=5.317682897 podStartE2EDuration="10.402184376s" totalImagesPullingTime="5.084501479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:38.401631094 +0000 UTC m=+45.979947850" watchObservedRunningTime="2026-07-23 19:01:38.402184376 +0000 UTC m=+45.980501134" Jul 23 19:01:38.430226 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.430160 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-54kmk" podStartSLOduration=3.343071976 podStartE2EDuration="8.430142129s" totalImagesPullingTime="5.087070153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:38.429659614 +0000 UTC m=+46.007976382" watchObservedRunningTime="2026-07-23 19:01:38.430142129 +0000 UTC m=+46.008458885" Jul 23 19:01:38.470907 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.470869 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:38.543902 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.543801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:38.653876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.653818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:38.654050 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.653877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:38.654050 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654006 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:38.654050 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654030 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.654005549 +0000 UTC m=+54.232322295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654072 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.654055143 +0000 UTC m=+54.232371892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.654011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654093 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654111 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.654113 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654158 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.654142167 +0000 UTC m=+54.232458911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654195 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 19:01:38.654235 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654227 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls podName:6c272505-3b03-497e-ba84-38743658d5b2 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.65421655 +0000 UTC m=+54.232533290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-46qdg" (UID: "6c272505-3b03-497e-ba84-38743658d5b2") : secret "samples-operator-tls" not found Jul 23 19:01:38.654730 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.654279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:38.654730 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654355 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:38.654730 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.654385 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.654375716 +0000 UTC m=+54.232692456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:38.755403 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:38.755364 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:38.755586 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.755570 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:38.755658 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:38.755630 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:46.755612074 +0000 UTC m=+54.333928827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:39.048182 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.048067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:39.144936 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.144908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:39.242983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.242947 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:39.306559 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.306491 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:01:39.306906 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.306888 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/0.log" Jul 23 19:01:39.307020 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.306927 2576 generic.go:356] "Generic (PLEG): container finished" podID="57d361da-ffa5-491c-8fa8-725e8a97513b" containerID="e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9" exitCode=255 Jul 23 19:01:39.307071 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.307018 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" event={"ID":"57d361da-ffa5-491c-8fa8-725e8a97513b","Type":"ContainerDied","Data":"e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9"} Jul 23 19:01:39.307071 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.307068 2576 scope.go:117] "RemoveContainer" containerID="bc3ec466fba64e8d63970973167cd996c4b009678a0108a3d11c3d90357f2417" Jul 23 19:01:39.307349 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.307334 2576 scope.go:117] "RemoveContainer" containerID="e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9" Jul 23 19:01:39.307544 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:39.307519 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-29gmr_openshift-console-operator(57d361da-ffa5-491c-8fa8-725e8a97513b)\"" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" podUID="57d361da-ffa5-491c-8fa8-725e8a97513b" Jul 23 19:01:39.655518 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.655484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:39.759517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.759490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:39.919491 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:39.919408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:40.011499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:40.011440 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:40.311760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:40.311726 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-wmcln" event={"ID":"cee55273-2d39-4ff5-81dc-534545313734","Type":"ContainerStarted","Data":"b04f159088f744ad38db72e5a76ca6a421930f2e04fa39dc5d70063ca5c98cec"} Jul 23 19:01:40.313289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:40.313270 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:01:40.313556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:40.313543 2576 scope.go:117] "RemoveContainer" containerID="e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9" Jul 23 19:01:40.313684 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:40.313668 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-29gmr_openshift-console-operator(57d361da-ffa5-491c-8fa8-725e8a97513b)\"" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" podUID="57d361da-ffa5-491c-8fa8-725e8a97513b" Jul 23 19:01:40.368588 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:40.368536 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-wmcln" podStartSLOduration=15.510991331 podStartE2EDuration="22.368519086s" totalImagesPullingTime="6.857527755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:40.337645355 +0000 UTC m=+47.915962110" watchObservedRunningTime="2026-07-23 19:01:40.368519086 +0000 UTC m=+47.946835841" Jul 23 19:01:41.201289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.201257 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:41.201647 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.201316 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:41.246552 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.246506 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-c8jcn"] Jul 23 19:01:41.265499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.265469 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c8jcn"] Jul 23 19:01:41.265671 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.265613 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.267896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.267875 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 19:01:41.268028 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.267876 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 19:01:41.268028 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.267902 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cdq4c\"" Jul 23 19:01:41.279954 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.279932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-data-volume\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.280044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.279959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwq5\" (UniqueName: \"kubernetes.io/projected/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-api-access-9zwq5\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.280044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.279981 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.280044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.280012 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-crio-socket\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.280147 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.280092 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.316734 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.316710 2576 scope.go:117] "RemoveContainer" containerID="e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9" Jul 23 19:01:41.316894 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.316862 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-29gmr_openshift-console-operator(57d361da-ffa5-491c-8fa8-725e8a97513b)\"" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" podUID="57d361da-ffa5-491c-8fa8-725e8a97513b" Jul 23 19:01:41.381036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.380998 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-data-volume\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwq5\" (UniqueName: \"kubernetes.io/projected/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-api-access-9zwq5\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381056 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381109 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-crio-socket\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.381145 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 19:01:41.381274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381191 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.381219 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:41.881200259 +0000 UTC m=+49.459516992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : configmap "kube-rbac-proxy" not found Jul 23 19:01:41.381567 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381359 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-crio-socket\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.381567 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.381548 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 19:01:41.381651 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.381606 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:41.881589819 +0000 UTC m=+49.459906554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : secret "insights-runtime-extractor-tls" not found Jul 23 19:01:41.381726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.381709 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-data-volume\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.392225 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.392198 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwq5\" (UniqueName: \"kubernetes.io/projected/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-api-access-9zwq5\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.885432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.885395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.885620 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:41.885473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:41.885620 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.885524 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 19:01:41.885620 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.885590 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:42.885575466 +0000 UTC m=+50.463892206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : configmap "kube-rbac-proxy" not found Jul 23 19:01:41.885776 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.885632 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 19:01:41.885776 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:41.885703 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:42.885684229 +0000 UTC m=+50.464000977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : secret "insights-runtime-extractor-tls" not found Jul 23 19:01:42.645722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:42.645642 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:42.750552 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:42.750517 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:42.893918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:42.893805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:42.894090 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:42.893884 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 19:01:42.894090 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:42.893991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:42.894090 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:42.894032 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:44.894018477 +0000 UTC m=+52.472335214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : configmap "kube-rbac-proxy" not found Jul 23 19:01:42.894216 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:42.894093 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 19:01:42.894216 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:42.894149 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:44.894133742 +0000 UTC m=+52.472450497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : secret "insights-runtime-extractor-tls" not found Jul 23 19:01:43.435769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:43.435742 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ct8h8_a956c0a0-2420-4040-913e-d714b4dc5418/dns-node-resolver/0.log" Jul 23 19:01:44.434287 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:44.434257 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4xq9v_f9582889-69da-4fc1-9340-3998fd371a39/node-ca/0.log" Jul 23 19:01:44.911643 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:44.911605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:44.911819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:44.911706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:44.911819 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:44.911725 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 19:01:44.911819 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:44.911790 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:48.911773207 +0000 UTC m=+56.490089960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : configmap "kube-rbac-proxy" not found Jul 23 19:01:44.911970 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:44.911879 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 19:01:44.915300 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:44.912274 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:48.912251999 +0000 UTC m=+56.490568797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : secret "insights-runtime-extractor-tls" not found Jul 23 19:01:45.230316 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:45.230237 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-54kmk_8e461165-4a68-4b19-9e61-10616c8eed5d/serve-healthcheck-canary/0.log" Jul 23 19:01:45.759400 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:45.759350 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:45.920888 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:45.920849 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.023319 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.023220 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.032671 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.032634 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-ks8td_8c8dddf0-949c-4228-a169-88a50e857fb2/kube-storage-version-migrator-operator/0.log" Jul 23 19:01:46.129762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.129721 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.353102 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.353001 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.454019 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.453977 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.538970 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.538915 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.728988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.728890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:01:46.728988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.728953 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:46.728988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.728977 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.728998 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.729016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729065 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729124 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729141 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls podName:e4dd8699-497d-4f9e-988f-f688f3aafd46 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.729120637 +0000 UTC m=+70.307437382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-s7krv" (UID: "e4dd8699-497d-4f9e-988f-f688f3aafd46") : secret "cluster-monitoring-operator-tls" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729144 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7764b7f96b-hwxtc: secret "image-registry-tls" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729161 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729183 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls podName:f3af9d7c-1310-47ea-8cc5-b2956180e0ff nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.729173195 +0000 UTC m=+70.307489974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls") pod "image-registry-7764b7f96b-hwxtc" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff") : secret "image-registry-tls" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729234 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.729217042 +0000 UTC m=+70.307533791 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : secret "router-metrics-certs-default" not found Jul 23 19:01:46.729274 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.729249 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle podName:5bfefa19-7c7e-4f52-a53e-a473d73137a5 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.729243465 +0000 UTC m=+70.307560198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle") pod "router-default-9c889d5d-pjpft" (UID: "5bfefa19-7c7e-4f52-a53e-a473d73137a5") : configmap references non-existent config key: service-ca.crt Jul 23 19:01:46.731441 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.731412 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c272505-3b03-497e-ba84-38743658d5b2-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-46qdg\" (UID: \"6c272505-3b03-497e-ba84-38743658d5b2\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:46.782273 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.782233 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" Jul 23 19:01:46.829646 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.829612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:01:46.829853 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.829798 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 19:01:46.829927 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:46.829892 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls podName:69e66d5f-e0db-426b-a333-a7f9243f1745 nodeName:}" failed. No retries permitted until 2026-07-23 19:02:02.829870962 +0000 UTC m=+70.408187712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls") pod "dns-default-rsv86" (UID: "69e66d5f-e0db-426b-a333-a7f9243f1745") : secret "dns-default-metrics-tls" not found Jul 23 19:01:46.835932 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.835892 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:46.929135 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.929101 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg"] Jul 23 19:01:46.934663 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:46.934635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:47.336670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:47.336638 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" event={"ID":"6c272505-3b03-497e-ba84-38743658d5b2","Type":"ContainerStarted","Data":"0828c819fb3e5ab70e14083d68aaa8e21e036e5de9928daeed36aaa90a4b8e1a"} Jul 23 19:01:47.359908 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:47.359868 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:47.863531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:47.863490 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:48.073188 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.073153 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:48.315116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.315073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:48.472139 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.472098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:48.947285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.947245 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:48.947716 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.947348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:48.947716 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:48.947489 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 19:01:48.947716 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:01:48.947555 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls podName:2ba43c2c-a266-490e-8537-fd1d88a0b7d9 nodeName:}" failed. No retries permitted until 2026-07-23 19:01:56.947539976 +0000 UTC m=+64.525856715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-c8jcn" (UID: "2ba43c2c-a266-490e-8537-fd1d88a0b7d9") : secret "insights-runtime-extractor-tls" not found Jul 23 19:01:48.947831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:48.947729 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:49.520461 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:49.520422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:49.670989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:49.670959 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:49.765996 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:49.765961 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:49.867743 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:49.867706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:50.139875 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.139770 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:50.275913 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.275873 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:50.350045 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.350013 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" event={"ID":"6c272505-3b03-497e-ba84-38743658d5b2","Type":"ContainerStarted","Data":"7741a6a51af6e27fab1cd591884b6c1fc066feb8ba426ed370f7e3d253ce34b6"} Jul 23 19:01:50.350216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.350090 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" event={"ID":"6c272505-3b03-497e-ba84-38743658d5b2","Type":"ContainerStarted","Data":"e822f21ecb0b94ac8aca5e902341acb8a80021a8b43352db6076fe8105dbd9e0"} Jul 23 19:01:50.371756 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.371707 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-46qdg" podStartSLOduration=23.581249846 podStartE2EDuration="26.371692211s" totalImagesPullingTime="2.790442365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:50.370520946 +0000 UTC m=+57.948837701" watchObservedRunningTime="2026-07-23 19:01:50.371692211 +0000 UTC m=+57.950008963" Jul 23 19:01:50.415279 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.415203 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:50.542318 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:50.542282 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:51.462058 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:51.462022 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:51.550800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:51.550766 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:51.653853 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:51.653807 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:51.715929 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:51.715824 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:51.863233 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:51.863204 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:52.262908 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:52.262878 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:52.338974 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:52.338938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:52.866946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:52.866912 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:53.480884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:53.480826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:53.661856 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:53.661788 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:54.061244 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:54.061198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:54.261506 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:54.261471 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:55.045379 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.045354 2576 scope.go:117] "RemoveContainer" containerID="e0dc8eb7f8298d17694b9d0f0c15da206deb95950ea71432ec8f0db2ac6c1dc9" Jul 23 19:01:55.365778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.365704 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:01:55.365778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.365748 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" event={"ID":"57d361da-ffa5-491c-8fa8-725e8a97513b","Type":"ContainerStarted","Data":"52191beb2790ff1b3d25af5ce387707109a4f4e54fe94af44d054146786fe327"} Jul 23 19:01:55.366269 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.366036 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:55.854706 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.854662 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:55.871995 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.871960 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" Jul 23 19:01:55.895168 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.895105 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-29gmr" podStartSLOduration=24.806341696 podStartE2EDuration="29.895082617s" totalImagesPullingTime="5.088740921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:01:55.399350563 +0000 UTC m=+62.977667319" watchObservedRunningTime="2026-07-23 19:01:55.895082617 +0000 UTC m=+63.473399374" Jul 23 19:01:55.955523 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:55.955492 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.077442 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.077406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.178101 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.178032 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.278799 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.278755 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.379191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.379156 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.486889 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.486792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.621359 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.621327 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:56.938818 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:56.938784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:57.013854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.013808 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:57.016259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.016226 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:57.016416 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.016267 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ba43c2c-a266-490e-8537-fd1d88a0b7d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-c8jcn\" (UID: \"2ba43c2c-a266-490e-8537-fd1d88a0b7d9\") " pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:57.178450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.178420 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cdq4c\"" Jul 23 19:01:57.185778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.185758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-c8jcn" Jul 23 19:01:57.240878 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.240850 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cgsqw" Jul 23 19:01:57.336366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.336337 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-c8jcn"] Jul 23 19:01:57.339197 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:01:57.339169 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba43c2c_a266_490e_8537_fd1d88a0b7d9.slice/crio-af48b0ccdca2dd4320614e8fe7d8c5b43bc7f8339030643e9f482183ee24a85e WatchSource:0}: Error finding container af48b0ccdca2dd4320614e8fe7d8c5b43bc7f8339030643e9f482183ee24a85e: Status 404 returned error can't find the container with id af48b0ccdca2dd4320614e8fe7d8c5b43bc7f8339030643e9f482183ee24a85e Jul 23 19:01:57.372148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.372118 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c8jcn" event={"ID":"2ba43c2c-a266-490e-8537-fd1d88a0b7d9","Type":"ContainerStarted","Data":"af48b0ccdca2dd4320614e8fe7d8c5b43bc7f8339030643e9f482183ee24a85e"} Jul 23 19:01:57.607829 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.607793 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:57.711365 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.711331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:01:57.807661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:01:57.807576 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:00.041792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.041754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:02:00.042344 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.041887 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:02:00.044679 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.044656 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 19:02:00.045165 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.045148 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 19:02:00.054512 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.054493 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a4014d-9b37-458d-b82d-4a29ef017488-metrics-certs\") pod \"network-metrics-daemon-kvmj8\" (UID: \"68a4014d-9b37-458d-b82d-4a29ef017488\") " pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:02:00.054512 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.054505 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c29a651-9e48-4ff9-842c-ccd602552695-original-pull-secret\") pod \"global-pull-secret-syncer-2q6nq\" (UID: \"8c29a651-9e48-4ff9-842c-ccd602552695\") " pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:02:00.143001 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.142957 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:02:00.145949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.145925 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 19:02:00.156012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.155975 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 19:02:00.156012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.155993 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5ztj\"" Jul 23 19:02:00.163889 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.163866 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kvmj8" Jul 23 19:02:00.166581 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.166560 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6cwq\" (UniqueName: \"kubernetes.io/projected/e28b1f13-a8c3-469d-a2fd-1348bc3645c3-kube-api-access-m6cwq\") pod \"network-check-target-j6lsv\" (UID: \"e28b1f13-a8c3-469d-a2fd-1348bc3645c3\") " pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:02:00.251601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.251569 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2q6nq" Jul 23 19:02:00.461334 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.461258 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8qs8\"" Jul 23 19:02:00.469395 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:00.469371 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:02:02.168081 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.168057 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j6lsv"] Jul 23 19:02:02.171064 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:02.171040 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28b1f13_a8c3_469d_a2fd_1348bc3645c3.slice/crio-e0f4fe2585075d1caedaf43382e183541fd254de4fbf075ff5f2ff5f73feadaf WatchSource:0}: Error finding container e0f4fe2585075d1caedaf43382e183541fd254de4fbf075ff5f2ff5f73feadaf: Status 404 returned error can't find the container with id e0f4fe2585075d1caedaf43382e183541fd254de4fbf075ff5f2ff5f73feadaf Jul 23 19:02:02.387312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.387288 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2q6nq"] Jul 23 19:02:02.391201 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:02.391173 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c29a651_9e48_4ff9_842c_ccd602552695.slice/crio-6164f4460d6ed103db61546702079538736eb876d7527af7d5fdb62f7c11edf4 WatchSource:0}: Error finding container 6164f4460d6ed103db61546702079538736eb876d7527af7d5fdb62f7c11edf4: Status 404 returned error can't find the container with id 6164f4460d6ed103db61546702079538736eb876d7527af7d5fdb62f7c11edf4 Jul 23 19:02:02.391684 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.391604 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kvmj8"] Jul 23 19:02:02.392779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.392743 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j6lsv" event={"ID":"e28b1f13-a8c3-469d-a2fd-1348bc3645c3","Type":"ContainerStarted","Data":"e0f4fe2585075d1caedaf43382e183541fd254de4fbf075ff5f2ff5f73feadaf"} Jul 23 19:02:02.394726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.394702 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c8jcn" event={"ID":"2ba43c2c-a266-490e-8537-fd1d88a0b7d9","Type":"ContainerStarted","Data":"8168c8ef26c22653ce26b34042a08a706ab59ccbb5dad21a432566f930371cc9"} Jul 23 19:02:02.395182 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:02.395147 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a4014d_9b37_458d_b82d_4a29ef017488.slice/crio-fdcda2d90657c0b0bb291c7a706137a2adee05bcf5aaa11819842dad62cfa85f WatchSource:0}: Error finding container fdcda2d90657c0b0bb291c7a706137a2adee05bcf5aaa11819842dad62cfa85f: Status 404 returned error can't find the container with id fdcda2d90657c0b0bb291c7a706137a2adee05bcf5aaa11819842dad62cfa85f Jul 23 19:02:02.767420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.767384 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:02:02.767605 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.767436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:02.767605 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.767459 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:02.767605 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.767488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:02.768300 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.768252 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bfefa19-7c7e-4f52-a53e-a473d73137a5-service-ca-bundle\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:02.770294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.770267 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4dd8699-497d-4f9e-988f-f688f3aafd46-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-s7krv\" (UID: \"e4dd8699-497d-4f9e-988f-f688f3aafd46\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:02:02.770389 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.770334 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bfefa19-7c7e-4f52-a53e-a473d73137a5-metrics-certs\") pod \"router-default-9c889d5d-pjpft\" (UID: \"5bfefa19-7c7e-4f52-a53e-a473d73137a5\") " pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:02.770389 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.770343 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"image-registry-7764b7f96b-hwxtc\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:02.793830 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.793787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:02.868038 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.867999 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:02:02.869892 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.869827 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:02.870571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.870553 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/69e66d5f-e0db-426b-a333-a7f9243f1745-metrics-tls\") pod \"dns-default-rsv86\" (UID: \"69e66d5f-e0db-426b-a333-a7f9243f1745\") " pod="openshift-dns/dns-default-rsv86" Jul 23 19:02:02.954489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.954451 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nv7r4\"" Jul 23 19:02:02.962624 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.962418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:02.977960 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.977729 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gnb95\"" Jul 23 19:02:02.986369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:02.985919 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" Jul 23 19:02:03.027013 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.026867 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n2skf\"" Jul 23 19:02:03.029383 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.029152 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:03.035691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.035348 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fgf46\"" Jul 23 19:02:03.044117 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.042594 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rsv86" Jul 23 19:02:03.044117 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.042797 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9"] Jul 23 19:02:03.050766 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.050686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:03.070811 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.070530 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.074434 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.074682 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.074921 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.075147 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.075502 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.075705 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 19:02:03.075919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.075917 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 23 19:02:03.081503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.081482 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9"] Jul 23 19:02:03.173060 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.172902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.173060 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.172952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.173060 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.172994 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.173060 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.173043 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.173632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.173084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.173632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.173133 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zm6f\" (UniqueName: \"kubernetes.io/projected/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-kube-api-access-9zm6f\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.219641 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.212243 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:02:03.260698 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.260666 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv"] Jul 23 19:02:03.264281 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:03.264237 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dd8699_497d_4f9e_988f_f688f3aafd46.slice/crio-a4330d3d7325509ebace153d6109cd80807667c85e22aa08ca184c685152a9b8 WatchSource:0}: Error finding container a4330d3d7325509ebace153d6109cd80807667c85e22aa08ca184c685152a9b8: Status 404 returned error can't find the container with id a4330d3d7325509ebace153d6109cd80807667c85e22aa08ca184c685152a9b8 Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273694 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zm6f\" (UniqueName: \"kubernetes.io/projected/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-kube-api-access-9zm6f\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.273872 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.278254 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.275758 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.287059 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.283298 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.287059 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.284597 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.287059 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.284687 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.287059 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.284745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-ca\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.289759 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.289715 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zm6f\" (UniqueName: \"kubernetes.io/projected/b733dd46-49c4-48d0-8ee3-b98c7cbc67c7-kube-api-access-9zm6f\") pod \"cluster-proxy-proxy-agent-7b84d75b5f-qx9b9\" (UID: \"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.321305 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.321273 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9c889d5d-pjpft"] Jul 23 19:02:03.327714 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:03.327595 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfefa19_7c7e_4f52_a53e_a473d73137a5.slice/crio-7c97cc17fc603be40a4310f7627ed9694c95295d765dfd40b7794b22fc06ad11 WatchSource:0}: Error finding container 7c97cc17fc603be40a4310f7627ed9694c95295d765dfd40b7794b22fc06ad11: Status 404 returned error can't find the container with id 7c97cc17fc603be40a4310f7627ed9694c95295d765dfd40b7794b22fc06ad11 Jul 23 19:02:03.343946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.343910 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:03.349287 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.348960 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rsv86"] Jul 23 19:02:03.356880 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:03.356849 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e66d5f_e0db_426b_a333_a7f9243f1745.slice/crio-275eded459e58b123b8dd50426595295e19cb55f3cf32b1b485e0cab735289c0 WatchSource:0}: Error finding container 275eded459e58b123b8dd50426595295e19cb55f3cf32b1b485e0cab735289c0: Status 404 returned error can't find the container with id 275eded459e58b123b8dd50426595295e19cb55f3cf32b1b485e0cab735289c0 Jul 23 19:02:03.403762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.403722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" event={"ID":"e4dd8699-497d-4f9e-988f-f688f3aafd46","Type":"ContainerStarted","Data":"a4330d3d7325509ebace153d6109cd80807667c85e22aa08ca184c685152a9b8"} Jul 23 19:02:03.405086 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.404998 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsv86" event={"ID":"69e66d5f-e0db-426b-a333-a7f9243f1745","Type":"ContainerStarted","Data":"275eded459e58b123b8dd50426595295e19cb55f3cf32b1b485e0cab735289c0"} Jul 23 19:02:03.406881 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.406651 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9c889d5d-pjpft" event={"ID":"5bfefa19-7c7e-4f52-a53e-a473d73137a5","Type":"ContainerStarted","Data":"7c97cc17fc603be40a4310f7627ed9694c95295d765dfd40b7794b22fc06ad11"} Jul 23 19:02:03.411860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.411814 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" event={"ID":"f3af9d7c-1310-47ea-8cc5-b2956180e0ff","Type":"ContainerStarted","Data":"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058"} Jul 23 19:02:03.412705 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.412647 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:03.412705 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.412677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" event={"ID":"f3af9d7c-1310-47ea-8cc5-b2956180e0ff","Type":"ContainerStarted","Data":"2b6acb48ea369552265c3370d047712bf5dcc9773e8c3e8b4e63ec3c16c124f5"} Jul 23 19:02:03.415206 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.415088 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2q6nq" event={"ID":"8c29a651-9e48-4ff9-842c-ccd602552695","Type":"ContainerStarted","Data":"6164f4460d6ed103db61546702079538736eb876d7527af7d5fdb62f7c11edf4"} Jul 23 19:02:03.417716 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.417660 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvmj8" event={"ID":"68a4014d-9b37-458d-b82d-4a29ef017488","Type":"ContainerStarted","Data":"fdcda2d90657c0b0bb291c7a706137a2adee05bcf5aaa11819842dad62cfa85f"} Jul 23 19:02:03.431185 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.431161 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" Jul 23 19:02:03.449694 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.449638 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:03.660198 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.659190 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" podStartSLOduration=67.659165497 podStartE2EDuration="1m7.659165497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:03.441237182 +0000 UTC m=+71.019553939" watchObservedRunningTime="2026-07-23 19:02:03.659165497 +0000 UTC m=+71.237482255" Jul 23 19:02:03.660198 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:03.659509 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9"] Jul 23 19:02:04.173568 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:04.173479 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:04.249897 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:04.249845 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb733dd46_49c4_48d0_8ee3_b98c7cbc67c7.slice/crio-17b948f2b59acac513fc56165f5d17bc287f72d908bd5be0be49bff4cf5be656 WatchSource:0}: Error finding container 17b948f2b59acac513fc56165f5d17bc287f72d908bd5be0be49bff4cf5be656: Status 404 returned error can't find the container with id 17b948f2b59acac513fc56165f5d17bc287f72d908bd5be0be49bff4cf5be656 Jul 23 19:02:04.427796 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:04.426871 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" event={"ID":"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7","Type":"ContainerStarted","Data":"17b948f2b59acac513fc56165f5d17bc287f72d908bd5be0be49bff4cf5be656"} Jul 23 19:02:04.429963 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:04.429879 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9c889d5d-pjpft" event={"ID":"5bfefa19-7c7e-4f52-a53e-a473d73137a5","Type":"ContainerStarted","Data":"b38dfb22b30a8aaa1b31e088f45cbe2c55dbca423e044b674a91c479c75f9d27"} Jul 23 19:02:05.030879 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.030779 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:05.033902 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.033875 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:05.036624 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.036530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:05.074431 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.073424 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9c889d5d-pjpft" podStartSLOduration=47.073403615 podStartE2EDuration="47.073403615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:04.458546848 +0000 UTC m=+72.036863607" watchObservedRunningTime="2026-07-23 19:02:05.073403615 +0000 UTC m=+72.651720371" Jul 23 19:02:05.138100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.138032 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:05.441920 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.440413 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvmj8" event={"ID":"68a4014d-9b37-458d-b82d-4a29ef017488","Type":"ContainerStarted","Data":"7c7e11f1f18e85e54c7ac9978ff7fc0e71b51ef0e2c1f43c0012392479052c1a"} Jul 23 19:02:05.441920 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.440492 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:05.442810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:05.442780 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9c889d5d-pjpft" Jul 23 19:02:07.590450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:07.590356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:07.661583 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:07.661546 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:13.473935 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.473870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c8jcn" event={"ID":"2ba43c2c-a266-490e-8537-fd1d88a0b7d9","Type":"ContainerStarted","Data":"51b3a11088a52a1b8a311e8a14af64714e28a9261e0545e5a33b35f8daedf95e"} Jul 23 19:02:13.475686 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.475656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsv86" event={"ID":"69e66d5f-e0db-426b-a333-a7f9243f1745","Type":"ContainerStarted","Data":"79c45595c0fe0926ef64dd519e491180e18085da926f3584d3cbc808cdbb7019"} Jul 23 19:02:13.475900 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.475883 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rsv86" event={"ID":"69e66d5f-e0db-426b-a333-a7f9243f1745","Type":"ContainerStarted","Data":"69c6306504ced7aa068bdec986c90fefff58b750d1ee3aabb8d9b26d60b2fc01"} Jul 23 19:02:13.477504 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.477478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2q6nq" event={"ID":"8c29a651-9e48-4ff9-842c-ccd602552695","Type":"ContainerStarted","Data":"8af2b395f2cdfb50fdaf3156729e4dfede558ff737c840161406fdce4f7e7989"} Jul 23 19:02:13.479603 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.479577 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j6lsv" event={"ID":"e28b1f13-a8c3-469d-a2fd-1348bc3645c3","Type":"ContainerStarted","Data":"670d5cf20160c2ae7fd87463deb3185faffe52604c0611d7860721f389229f27"} Jul 23 19:02:13.479709 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.479663 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:02:13.481744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.481694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kvmj8" event={"ID":"68a4014d-9b37-458d-b82d-4a29ef017488","Type":"ContainerStarted","Data":"a39c81568d5791f2079eb8fdae99fd9cba994bd1a9ade5eb8507cf30bf82f2b0"} Jul 23 19:02:13.483450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.483415 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" event={"ID":"e4dd8699-497d-4f9e-988f-f688f3aafd46","Type":"ContainerStarted","Data":"0e0a10d7063a42ab2a2767bcc3eab3884062e44249f733f5239df2e0334222df"} Jul 23 19:02:13.485938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.485911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" event={"ID":"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7","Type":"ContainerStarted","Data":"499adb44d8b57eb9656868a78d94e7e8f29fb108730e089531dbb4e04e996428"} Jul 23 19:02:13.486031 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.485951 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" event={"ID":"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7","Type":"ContainerStarted","Data":"269ff1fd047428f0f046887c597ef4c6dc786cbb2e459572c02a7463541c9761"} Jul 23 19:02:13.486031 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.485971 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" event={"ID":"b733dd46-49c4-48d0-8ee3-b98c7cbc67c7","Type":"ContainerStarted","Data":"612416578d20095bd7e53eea0b1cd18abc1107e4c285400fa7003c642388be97"} Jul 23 19:02:13.501385 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.501344 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rsv86" podStartSLOduration=33.722315653 podStartE2EDuration="43.501331596s" totalImagesPullingTime="9.779015943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.500065048 +0000 UTC m=+81.078381804" watchObservedRunningTime="2026-07-23 19:02:13.501331596 +0000 UTC m=+81.079648350" Jul 23 19:02:13.524030 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.523980 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kvmj8" podStartSLOduration=76.310136704 podStartE2EDuration="1m18.523962484s" totalImagesPullingTime="2.21382578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.523462743 +0000 UTC m=+81.101779499" watchObservedRunningTime="2026-07-23 19:02:13.523962484 +0000 UTC m=+81.102279239" Jul 23 19:02:13.559133 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.559072 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b84d75b5f-qx9b9" podStartSLOduration=1.649416435 podStartE2EDuration="10.559051237s" totalImagesPullingTime="8.909634802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:02:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.557958092 +0000 UTC m=+81.136274851" watchObservedRunningTime="2026-07-23 19:02:13.559051237 +0000 UTC m=+81.137367999" Jul 23 19:02:13.587690 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.587625 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2q6nq" podStartSLOduration=66.842408334 podStartE2EDuration="1m17.587605285s" totalImagesPullingTime="10.745196951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.5865223 +0000 UTC m=+81.164839054" watchObservedRunningTime="2026-07-23 19:02:13.587605285 +0000 UTC m=+81.165922041" Jul 23 19:02:13.610159 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.610097 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-s7krv" podStartSLOduration=46.928856882 podStartE2EDuration="55.610079642s" totalImagesPullingTime="8.68122276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.609551445 +0000 UTC m=+81.187868217" watchObservedRunningTime="2026-07-23 19:02:13.610079642 +0000 UTC m=+81.188396403" Jul 23 19:02:13.640677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.640565 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j6lsv" podStartSLOduration=66.674739569 podStartE2EDuration="1m17.640542073s" totalImagesPullingTime="10.965802504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:00:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:13.639266382 +0000 UTC m=+81.217583139" watchObservedRunningTime="2026-07-23 19:02:13.640542073 +0000 UTC m=+81.218858832" Jul 23 19:02:13.657488 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.657450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:13.788479 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.788394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:13.930368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:13.930269 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:14.138713 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.138673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:14.238815 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.238715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:14.339853 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.339802 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:14.491358 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.491183 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rsv86" Jul 23 19:02:14.628596 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.628558 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:14.755398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:14.755357 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:15.023050 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:15.022950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:16.595302 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:16.595164 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:16.662272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:16.662228 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:17.505309 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:17.505272 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-c8jcn" event={"ID":"2ba43c2c-a266-490e-8537-fd1d88a0b7d9","Type":"ContainerStarted","Data":"6eeff9b1c708a8662d36810729b353c9ace94136ca02346e0f63c1bf1a906639"} Jul 23 19:02:17.529768 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:17.529716 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-c8jcn" podStartSLOduration=17.040690751 podStartE2EDuration="36.529697783s" totalImagesPullingTime="19.489007032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:01:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:17.529268919 +0000 UTC m=+85.107585674" watchObservedRunningTime="2026-07-23 19:02:17.529697783 +0000 UTC m=+85.108014535" Jul 23 19:02:17.810527 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:17.810419 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:17.898537 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:17.898494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:18.009061 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:18.009004 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:18.986697 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:18.986660 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:19.046250 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:19.046205 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:21.771401 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:21.771372 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:22.966617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:22.966578 2576 patch_prober.go:28] interesting pod/image-registry-7764b7f96b-hwxtc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 19:02:22.967012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:22.966653 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 19:02:24.049299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.049262 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.271043 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.270999 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.380170 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.380068 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.426656 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.426622 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6glc9"] Jul 23 19:02:24.432881 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.432855 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.434441 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.434423 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:24.435897 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.435883 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 19:02:24.435965 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.435884 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 19:02:24.436590 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.436569 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 19:02:24.436699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.436601 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 19:02:24.436699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.436615 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b6jgk\"" Jul 23 19:02:24.439601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439439 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-sys\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439479 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-root\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-textfile\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439586 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-metrics-client-ca\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439812 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jnqn\" (UniqueName: \"kubernetes.io/projected/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-kube-api-access-6jnqn\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439812 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439812 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-accelerators-collector-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.439812 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.439734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-tls\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.440395 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.440373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-wtmp\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.472448 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.472415 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.497604 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.497573 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rsv86" Jul 23 19:02:24.540811 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.540774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-textfile\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.540811 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.540818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-metrics-client-ca\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541054 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.540863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jnqn\" (UniqueName: \"kubernetes.io/projected/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-kube-api-access-6jnqn\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541054 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.540973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541054 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-accelerators-collector-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-tls\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541240 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-wtmp\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541275 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-sys\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-root\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541424 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541383 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-root\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541424 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541404 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-wtmp\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541424 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541419 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-sys\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.541861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.541824 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-textfile\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.542196 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.542171 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-metrics-client-ca\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.542307 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.542221 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-accelerators-collector-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.543881 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.543860 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.544163 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.544142 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-node-exporter-tls\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.551152 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.551122 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jnqn\" (UniqueName: \"kubernetes.io/projected/1b9ef68b-711e-4bcd-b405-99db0b1ce37e-kube-api-access-6jnqn\") pod \"node-exporter-6glc9\" (UID: \"1b9ef68b-711e-4bcd-b405-99db0b1ce37e\") " pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.572494 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.572457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.673550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.673467 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.741933 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.741901 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6glc9" Jul 23 19:02:24.757297 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:02:24.757267 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9ef68b_711e_4bcd_b405_99db0b1ce37e.slice/crio-299e6c349a7f3cb3c9e2bb374c7a06637376c481c003552978398165f4fe42d7 WatchSource:0}: Error finding container 299e6c349a7f3cb3c9e2bb374c7a06637376c481c003552978398165f4fe42d7: Status 404 returned error can't find the container with id 299e6c349a7f3cb3c9e2bb374c7a06637376c481c003552978398165f4fe42d7 Jul 23 19:02:24.769434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.769409 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:24.891135 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:24.891101 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.046789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.046759 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.146636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.146603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.318677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.318593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.415406 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.415365 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.536072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.536033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6glc9" event={"ID":"1b9ef68b-711e-4bcd-b405-99db0b1ce37e","Type":"ContainerStarted","Data":"299e6c349a7f3cb3c9e2bb374c7a06637376c481c003552978398165f4fe42d7"} Jul 23 19:02:25.672504 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.672414 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.748732 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.748694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:25.857713 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:25.857669 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:26.968101 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:26.968068 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:02:27.067769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.067732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.171426 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.171389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.376342 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.376313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.446587 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.446524 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.544741 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.544643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.545100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.545077 2576 generic.go:356] "Generic (PLEG): container finished" podID="1b9ef68b-711e-4bcd-b405-99db0b1ce37e" containerID="d4e3e9bbb381392965b33f37e1c7b15dd13d260088eced56a36c9b4795fb5429" exitCode=0 Jul 23 19:02:27.545191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.545159 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6glc9" event={"ID":"1b9ef68b-711e-4bcd-b405-99db0b1ce37e","Type":"ContainerDied","Data":"d4e3e9bbb381392965b33f37e1c7b15dd13d260088eced56a36c9b4795fb5429"} Jul 23 19:02:27.827651 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.827565 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:27.927377 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:27.927345 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:28.169442 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.169353 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:28.261595 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.261561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:28.367370 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.367333 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:28.554062 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.554031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6glc9" event={"ID":"1b9ef68b-711e-4bcd-b405-99db0b1ce37e","Type":"ContainerStarted","Data":"7e91869acd00543be0d62eda9b2f75bcc285dd9c05e449401f6af4d0bbe3de1b"} Jul 23 19:02:28.554230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.554091 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6glc9" event={"ID":"1b9ef68b-711e-4bcd-b405-99db0b1ce37e","Type":"ContainerStarted","Data":"cb646f324d7ac8a404300c842d043a832176afe0e64b2c5bf8d506df49a0ebce"} Jul 23 19:02:28.577929 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:28.577884 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6glc9" podStartSLOduration=2.136781168 podStartE2EDuration="4.577868678s" totalImagesPullingTime="2.376260613s" totalInitContainerRuntime="64.826897ms" isStatefulPod=true podCreationTimestamp="2026-07-23 19:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:02:28.576974171 +0000 UTC m=+96.155290928" watchObservedRunningTime="2026-07-23 19:02:28.577868678 +0000 UTC m=+96.156185432" Jul 23 19:02:29.067346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:29.067311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:29.171730 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:29.171692 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:29.275723 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:29.275679 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:29.367475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:29.367379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:30.918698 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:30.918659 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:31.021274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:31.021241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:31.518109 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:31.518070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:31.617128 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:31.617095 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:33.204276 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:33.204241 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:33.268160 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:33.268130 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:35.351098 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:35.351054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:35.429007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:35.428920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:38.292659 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:38.292620 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:38.428397 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:38.428362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:39.598447 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:39.598408 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:40.587227 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:40.587190 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:40.649364 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:40.649330 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:41.766319 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:41.766281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:41.868437 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:41.868403 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:41.968805 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:41.968751 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:42.345694 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:42.345659 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:42.413521 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:42.413477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:44.493729 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:44.493700 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j6lsv" Jul 23 19:02:46.362032 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:46.361991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:46.670041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:46.669948 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:02:49.629592 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:49.629555 2576 generic.go:356] "Generic (PLEG): container finished" podID="aa7a3cf6-e8be-4448-a8b5-14524115c6b4" containerID="fad5ae0c90708f2783f189c9e392ffdb0b7f25806b7e038e53004e810f56df91" exitCode=0 Jul 23 19:02:49.629980 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:49.629613 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" event={"ID":"aa7a3cf6-e8be-4448-a8b5-14524115c6b4","Type":"ContainerDied","Data":"fad5ae0c90708f2783f189c9e392ffdb0b7f25806b7e038e53004e810f56df91"} Jul 23 19:02:49.629980 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:49.629900 2576 scope.go:117] "RemoveContainer" containerID="fad5ae0c90708f2783f189c9e392ffdb0b7f25806b7e038e53004e810f56df91" Jul 23 19:02:50.634946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:50.634913 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-wj64c" event={"ID":"aa7a3cf6-e8be-4448-a8b5-14524115c6b4","Type":"ContainerStarted","Data":"32e4acb71071857f88fa7277f04d19aee9f8f588b0ded76a3c81f47483db72f5"} Jul 23 19:02:51.640278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:51.640245 2576 generic.go:356] "Generic (PLEG): container finished" podID="cee55273-2d39-4ff5-81dc-534545313734" containerID="b04f159088f744ad38db72e5a76ca6a421930f2e04fa39dc5d70063ca5c98cec" exitCode=0 Jul 23 19:02:51.640648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:51.640285 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-wmcln" event={"ID":"cee55273-2d39-4ff5-81dc-534545313734","Type":"ContainerDied","Data":"b04f159088f744ad38db72e5a76ca6a421930f2e04fa39dc5d70063ca5c98cec"} Jul 23 19:02:51.640648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:51.640604 2576 scope.go:117] "RemoveContainer" containerID="b04f159088f744ad38db72e5a76ca6a421930f2e04fa39dc5d70063ca5c98cec" Jul 23 19:02:51.990738 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:51.990646 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerName="registry" containerID="cri-o://1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058" gracePeriod=30 Jul 23 19:02:52.271088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.270919 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:52.369889 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.369798 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.369921 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5bg9\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.369960 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370019 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370053 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370081 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370104 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370158 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets\") pod \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\" (UID: \"f3af9d7c-1310-47ea-8cc5-b2956180e0ff\") " Jul 23 19:02:52.370467 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370441 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:02:52.370653 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.370622 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:02:52.372944 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.372868 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 19:02:52.372944 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.372924 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9" (OuterVolumeSpecName: "kube-api-access-f5bg9") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "kube-api-access-f5bg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:02:52.373148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.373038 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:02:52.373148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.373115 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:02:52.373148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.373124 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 19:02:52.379443 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.379408 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f3af9d7c-1310-47ea-8cc5-b2956180e0ff" (UID: "f3af9d7c-1310-47ea-8cc5-b2956180e0ff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:02:52.471245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471211 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-image-registry-private-configuration\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471242 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-certificates\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471253 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-installation-pull-secrets\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471264 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-bound-sa-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471273 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5bg9\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-kube-api-access-f5bg9\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471282 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-trusted-ca\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471289 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-registry-tls\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.471475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.471297 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f3af9d7c-1310-47ea-8cc5-b2956180e0ff-ca-trust-extracted\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:02:52.647661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.647627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-wmcln" event={"ID":"cee55273-2d39-4ff5-81dc-534545313734","Type":"ContainerStarted","Data":"e5671201dd7dfde525a443b9b153e15685babd11225cfb7b0a6f2bb57f259f3e"} Jul 23 19:02:52.648873 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.648831 2576 generic.go:356] "Generic (PLEG): container finished" podID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerID="1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058" exitCode=0 Jul 23 19:02:52.648973 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.648888 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" Jul 23 19:02:52.648973 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.648926 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" event={"ID":"f3af9d7c-1310-47ea-8cc5-b2956180e0ff","Type":"ContainerDied","Data":"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058"} Jul 23 19:02:52.649040 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.648971 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7764b7f96b-hwxtc" event={"ID":"f3af9d7c-1310-47ea-8cc5-b2956180e0ff","Type":"ContainerDied","Data":"2b6acb48ea369552265c3370d047712bf5dcc9773e8c3e8b4e63ec3c16c124f5"} Jul 23 19:02:52.649040 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.648993 2576 scope.go:117] "RemoveContainer" containerID="1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058" Jul 23 19:02:52.658135 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.658116 2576 scope.go:117] "RemoveContainer" containerID="1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058" Jul 23 19:02:52.658402 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:02:52.658380 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058\": container with ID starting with 1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058 not found: ID does not exist" containerID="1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058" Jul 23 19:02:52.658451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.658410 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058"} err="failed to get container status \"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058\": rpc error: code = NotFound desc = could not find container \"1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058\": container with ID starting with 1392be81c8543570f4ef1c35eef197b3b8b5c89d83efc93b715f8f06d3c97058 not found: ID does not exist" Jul 23 19:02:52.694817 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.694788 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:02:52.698985 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:52.698962 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7764b7f96b-hwxtc"] Jul 23 19:02:53.046447 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:53.046415 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" path="/var/lib/kubelet/pods/f3af9d7c-1310-47ea-8cc5-b2956180e0ff/volumes" Jul 23 19:02:53.653569 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:53.653536 2576 generic.go:356] "Generic (PLEG): container finished" podID="8c8dddf0-949c-4228-a169-88a50e857fb2" containerID="32c23383ceb43d151133c3086df5b6a9b63f0d66fcca130c73610e0243c2f5bb" exitCode=0 Jul 23 19:02:53.654035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:53.653607 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" event={"ID":"8c8dddf0-949c-4228-a169-88a50e857fb2","Type":"ContainerDied","Data":"32c23383ceb43d151133c3086df5b6a9b63f0d66fcca130c73610e0243c2f5bb"} Jul 23 19:02:53.654035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:53.653933 2576 scope.go:117] "RemoveContainer" containerID="32c23383ceb43d151133c3086df5b6a9b63f0d66fcca130c73610e0243c2f5bb" Jul 23 19:02:54.658754 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:02:54.658715 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-ks8td" event={"ID":"8c8dddf0-949c-4228-a169-88a50e857fb2","Type":"ContainerStarted","Data":"b02ff706b92be015bd12078a4e117bd075419cb058f7760f25143aa0a4d54e02"} Jul 23 19:03:01.347091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:01.347048 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:01.415475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:01.415433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:09.819646 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:09.819603 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:19.081616 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:19.081573 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:41.004535 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:41.004494 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:43.935696 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:43.935657 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:44.110456 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:44.110418 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:52.656182 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:52.656094 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:57.380035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:57.379993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:57.449714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:57.449675 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:59.107193 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:59.107163 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:03:59.177548 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:03:59.177513 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:11.121757 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:11.121720 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:11.235163 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:11.235126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:11.305702 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:11.305661 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:16.384150 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:16.384111 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:16.483385 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:16.483331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:16.548990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:16.548951 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:22.563469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:22.563422 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:22.637102 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:22.637067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:04:29.754205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:04:29.754168 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:05:00.109607 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:00.109567 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:05:00.307580 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:00.307541 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:05:11.306397 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:11.306351 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:05:11.393077 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:11.393037 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:05:52.896340 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:52.896312 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:05:52.896340 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:52.896324 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:05:52.910454 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:52.910416 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 19:05:53.796566 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:05:53.796525 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:41.401211 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:41.401176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:43.338747 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.338708 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6"] Jul 23 19:06:43.339151 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.339121 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerName="registry" Jul 23 19:06:43.339151 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.339141 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerName="registry" Jul 23 19:06:43.339241 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.339220 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3af9d7c-1310-47ea-8cc5-b2956180e0ff" containerName="registry" Jul 23 19:06:43.342104 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.342083 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.345328 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.345305 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 19:06:43.345939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.345923 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:06:43.346031 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.345952 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-98wzh\"" Jul 23 19:06:43.371003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.370966 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6"] Jul 23 19:06:43.405678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.405636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpdgl\" (UniqueName: \"kubernetes.io/projected/ba378de9-9f0d-457c-81a8-792c89cea14b-kube-api-access-jpdgl\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.405893 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.405755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba378de9-9f0d-457c-81a8-792c89cea14b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.506416 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.506375 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpdgl\" (UniqueName: \"kubernetes.io/projected/ba378de9-9f0d-457c-81a8-792c89cea14b-kube-api-access-jpdgl\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.506597 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.506448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba378de9-9f0d-457c-81a8-792c89cea14b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.506769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.506754 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba378de9-9f0d-457c-81a8-792c89cea14b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.517278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.517252 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpdgl\" (UniqueName: \"kubernetes.io/projected/ba378de9-9f0d-457c-81a8-792c89cea14b-kube-api-access-jpdgl\") pod \"cert-manager-operator-controller-manager-55c444bf86-qdxr6\" (UID: \"ba378de9-9f0d-457c-81a8-792c89cea14b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.651116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.651013 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" Jul 23 19:06:43.791855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.791784 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6"] Jul 23 19:06:43.794858 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:06:43.794794 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba378de9_9f0d_457c_81a8_792c89cea14b.slice/crio-90d7230568bf2a7419c53676445b19492d2f6588bab39a3d607bc9a3700ba1a7 WatchSource:0}: Error finding container 90d7230568bf2a7419c53676445b19492d2f6588bab39a3d607bc9a3700ba1a7: Status 404 returned error can't find the container with id 90d7230568bf2a7419c53676445b19492d2f6588bab39a3d607bc9a3700ba1a7 Jul 23 19:06:43.797419 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:43.797396 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 19:06:44.455028 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:44.454992 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" event={"ID":"ba378de9-9f0d-457c-81a8-792c89cea14b","Type":"ContainerStarted","Data":"90d7230568bf2a7419c53676445b19492d2f6588bab39a3d607bc9a3700ba1a7"} Jul 23 19:06:44.470938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:44.470906 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:45.226103 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:45.226063 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:45.329769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:45.329734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:48.372636 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:48.372593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:48.472983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:48.472946 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:48.478340 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:48.478313 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" event={"ID":"ba378de9-9f0d-457c-81a8-792c89cea14b","Type":"ContainerStarted","Data":"71e381c665e17e93af05fde6831cef217f59a788b5c745f0314cd4887a87cf39"} Jul 23 19:06:48.504245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:48.504183 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qdxr6" podStartSLOduration=1.561746754 podStartE2EDuration="5.504165775s" totalImagesPullingTime="3.942419021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:06:48.50310249 +0000 UTC m=+356.081419246" watchObservedRunningTime="2026-07-23 19:06:48.504165775 +0000 UTC m=+356.082482531" Jul 23 19:06:49.047718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:49.047683 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:49.145544 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:49.145504 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:50.860959 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:50.860920 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:50.944292 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:50.944254 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:57.859939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:57.859891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:57.959963 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:57.959933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:59.644309 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:59.644247 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:59.902298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:59.902212 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:06:59.991041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:06:59.990999 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:05.125932 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:05.125894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:05.242815 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:05.242740 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:05.528760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:05.528731 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:05.630025 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:05.629982 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:07.534963 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:07.534926 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:22.423962 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:22.423925 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:22.665230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:22.665191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:23.054884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.054828 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:23.417219 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.417129 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:23.514350 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.514311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:23.587696 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.587661 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw"] Jul 23 19:07:23.592217 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.592199 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.596977 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.596952 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 19:07:23.597119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.597075 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 19:07:23.597119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.597078 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 19:07:23.597119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.597113 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wh78r\"" Jul 23 19:07:23.597493 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.597478 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 19:07:23.617818 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.617789 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw"] Jul 23 19:07:23.721245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.721163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxtkc\" (UniqueName: \"kubernetes.io/projected/5ab76fc3-92e8-4941-a838-fed7e74da552-kube-api-access-sxtkc\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.721245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.721199 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.721245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.721225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.721454 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.721287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.721454 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.721393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.821794 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.821739 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.821794 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.821803 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.822090 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.821904 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.822090 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.821942 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxtkc\" (UniqueName: \"kubernetes.io/projected/5ab76fc3-92e8-4941-a838-fed7e74da552-kube-api-access-sxtkc\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.822090 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.821970 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.822298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.822275 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-manifests\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.824201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.824177 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ab76fc3-92e8-4941-a838-fed7e74da552-tmp\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.824328 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.824311 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-webhook-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.824511 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.824490 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5ab76fc3-92e8-4941-a838-fed7e74da552-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.833770 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.833745 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxtkc\" (UniqueName: \"kubernetes.io/projected/5ab76fc3-92e8-4941-a838-fed7e74da552-kube-api-access-sxtkc\") pod \"opendatahub-operator-controller-manager-6657c8cb8d-cw9gw\" (UID: \"5ab76fc3-92e8-4941-a838-fed7e74da552\") " pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.862651 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.862619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:23.902472 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.902444 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:23.964883 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:23.964781 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:24.047127 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.047102 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw"] Jul 23 19:07:24.049557 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:07:24.049531 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab76fc3_92e8_4941_a838_fed7e74da552.slice/crio-5139b9d7dc7ba410bdc5ecd7df9ffb4904adf7246272a8b21fc52504b8267b5e WatchSource:0}: Error finding container 5139b9d7dc7ba410bdc5ecd7df9ffb4904adf7246272a8b21fc52504b8267b5e: Status 404 returned error can't find the container with id 5139b9d7dc7ba410bdc5ecd7df9ffb4904adf7246272a8b21fc52504b8267b5e Jul 23 19:07:24.070978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.070948 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:24.171939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.171907 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:24.274939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.274908 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:24.372698 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.372656 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:24.612699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:24.612614 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" event={"ID":"5ab76fc3-92e8-4941-a838-fed7e74da552","Type":"ContainerStarted","Data":"5139b9d7dc7ba410bdc5ecd7df9ffb4904adf7246272a8b21fc52504b8267b5e"} Jul 23 19:07:27.626170 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:27.626130 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" event={"ID":"5ab76fc3-92e8-4941-a838-fed7e74da552","Type":"ContainerStarted","Data":"85762f0f396c6aef31c10524e91d931334dcc3f9f4842026d62e7dace6e2a9a8"} Jul 23 19:07:28.631520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:28.631484 2576 generic.go:356] "Generic (PLEG): container finished" podID="5ab76fc3-92e8-4941-a838-fed7e74da552" containerID="85762f0f396c6aef31c10524e91d931334dcc3f9f4842026d62e7dace6e2a9a8" exitCode=0 Jul 23 19:07:28.631929 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:28.631573 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" event={"ID":"5ab76fc3-92e8-4941-a838-fed7e74da552","Type":"ContainerDied","Data":"85762f0f396c6aef31c10524e91d931334dcc3f9f4842026d62e7dace6e2a9a8"} Jul 23 19:07:29.638296 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:29.638264 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" event={"ID":"5ab76fc3-92e8-4941-a838-fed7e74da552","Type":"ContainerStarted","Data":"7f8647ce34cc198dea6970f6208d358dde5a296a3388f50f64e8207d9c9ca5c2"} Jul 23 19:07:29.638728 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:29.638364 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:29.668063 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:29.668013 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" podStartSLOduration=2.287744189 podStartE2EDuration="6.667997267s" totalImagesPullingTime="3.370867472s" totalInitContainerRuntime="1.009385606s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:29.66618297 +0000 UTC m=+397.244499728" watchObservedRunningTime="2026-07-23 19:07:29.667997267 +0000 UTC m=+397.246314022" Jul 23 19:07:40.645227 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:40.645196 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6657c8cb8d-cw9gw" Jul 23 19:07:42.685166 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.685124 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:42.796607 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.796561 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:42.809649 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.809621 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-779d54d68-kghst"] Jul 23 19:07:42.817798 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.817776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.821162 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.821140 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 19:07:42.821328 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.821309 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 19:07:42.821940 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.821925 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 19:07:42.822066 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.822048 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 19:07:42.822434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.822418 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 19:07:42.825792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.825777 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xgzb6\"" Jul 23 19:07:42.834303 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.834278 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779d54d68-kghst"] Jul 23 19:07:42.866153 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.866118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.866313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.866160 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39b15187-ad67-41b9-88e4-828d8e98101d-manager-config\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.866313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.866191 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t8lb\" (UniqueName: \"kubernetes.io/projected/39b15187-ad67-41b9-88e4-828d8e98101d-kube-api-access-4t8lb\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.866313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.866252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-metrics-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.967556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.967469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.967556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.967516 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39b15187-ad67-41b9-88e4-828d8e98101d-manager-config\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.967752 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.967564 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t8lb\" (UniqueName: \"kubernetes.io/projected/39b15187-ad67-41b9-88e4-828d8e98101d-kube-api-access-4t8lb\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.967752 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.967591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-metrics-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.968309 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.968281 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/39b15187-ad67-41b9-88e4-828d8e98101d-manager-config\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.970073 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.970051 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-metrics-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.970179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.970109 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39b15187-ad67-41b9-88e4-828d8e98101d-cert\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:42.987271 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:42.987244 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t8lb\" (UniqueName: \"kubernetes.io/projected/39b15187-ad67-41b9-88e4-828d8e98101d-kube-api-access-4t8lb\") pod \"lws-controller-manager-779d54d68-kghst\" (UID: \"39b15187-ad67-41b9-88e4-828d8e98101d\") " pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:43.127022 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:43.126987 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:43.274709 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:43.274683 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779d54d68-kghst"] Jul 23 19:07:43.276789 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:07:43.276761 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b15187_ad67_41b9_88e4_828d8e98101d.slice/crio-6440e34ad17c2b9b373baac3cbaedfd0ec017f462f998cec11136217491f45ed WatchSource:0}: Error finding container 6440e34ad17c2b9b373baac3cbaedfd0ec017f462f998cec11136217491f45ed: Status 404 returned error can't find the container with id 6440e34ad17c2b9b373baac3cbaedfd0ec017f462f998cec11136217491f45ed Jul 23 19:07:43.693733 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:43.693614 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" event={"ID":"39b15187-ad67-41b9-88e4-828d8e98101d","Type":"ContainerStarted","Data":"6440e34ad17c2b9b373baac3cbaedfd0ec017f462f998cec11136217491f45ed"} Jul 23 19:07:43.773738 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:43.773705 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:43.897051 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:43.896871 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:44.601070 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:44.601017 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:45.117653 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:45.117605 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:45.220926 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:45.220889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:45.325685 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:45.325643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:45.421100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:45.421012 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:46.096722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.096673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:46.226854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.226808 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:46.343128 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.343083 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:46.472312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.472223 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:46.709581 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.709545 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" event={"ID":"39b15187-ad67-41b9-88e4-828d8e98101d","Type":"ContainerStarted","Data":"09053f4e183f7b2351a8cb59e52b76c394a2f5236b85a9b243033fa5dde47514"} Jul 23 19:07:46.709751 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.709679 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:46.770873 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:46.770806 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" podStartSLOduration=2.381141017 podStartE2EDuration="4.770787725s" totalImagesPullingTime="2.389646708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:46.768963667 +0000 UTC m=+414.347280424" watchObservedRunningTime="2026-07-23 19:07:46.770787725 +0000 UTC m=+414.349104479" Jul 23 19:07:48.326975 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:48.326934 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:51.794471 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:51.794424 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:51.915486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:51.915457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:52.758362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.758329 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:52.775673 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.775644 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv"] Jul 23 19:07:52.779063 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.779046 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.788547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.788529 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 19:07:52.788657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.788632 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d869v\"" Jul 23 19:07:52.853088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j59ql\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-kube-api-access-j59ql\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/014d8ce8-f7f0-4b96-990c-e69e311ded73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853173 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853203 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853229 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853287 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.853510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.853397 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.858238 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.858208 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv"] Jul 23 19:07:52.861984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.861960 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn"] Jul 23 19:07:52.865092 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.865077 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.890036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.890009 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 19:07:52.890191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.890068 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 19:07:52.890191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.890108 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 19:07:52.890373 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.890358 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pn7xv\"" Jul 23 19:07:52.931366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.931338 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn"] Jul 23 19:07:52.954617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2217e5d1-03bb-444a-a812-fac50591dcee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.954617 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954618 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954734 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8krsq\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-kube-api-access-8krsq\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.954806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954783 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954901 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.955125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.954982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j59ql\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-kube-api-access-j59ql\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/014d8ce8-f7f0-4b96-990c-e69e311ded73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955121 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955126 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955147 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:52.955362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955178 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955564 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955385 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955564 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955468 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.955564 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.955555 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.957306 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.957282 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.957789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.957757 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:52.966387 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.966364 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 19:07:52.975821 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:52.975801 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/014d8ce8-f7f0-4b96-990c-e69e311ded73-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:53.007463 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.007425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j59ql\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-kube-api-access-j59ql\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:53.007774 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.007754 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/014d8ce8-f7f0-4b96-990c-e69e311ded73-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678rksmv\" (UID: \"014d8ce8-f7f0-4b96-990c-e69e311ded73\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:53.056471 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056471 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2217e5d1-03bb-444a-a812-fac50591dcee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056471 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056451 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056742 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056742 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056742 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056662 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8krsq\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-kube-api-access-8krsq\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.056742 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.056710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.057323 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.057297 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.058865 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.058812 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.058972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.058891 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.059010 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.058998 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2217e5d1-03bb-444a-a812-fac50591dcee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.059194 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.059175 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2217e5d1-03bb-444a-a812-fac50591dcee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.078921 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.078898 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.090640 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.090617 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8krsq\" (UniqueName: \"kubernetes.io/projected/2217e5d1-03bb-444a-a812-fac50591dcee-kube-api-access-8krsq\") pod \"istiod-openshift-gateway-94bb8fbfd-lt8fn\" (UID: \"2217e5d1-03bb-444a-a812-fac50591dcee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.102699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.102674 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d869v\"" Jul 23 19:07:53.108921 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.108899 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:53.174418 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.174375 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:07:53.302626 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.302591 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv"] Jul 23 19:07:53.306108 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:07:53.306078 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014d8ce8_f7f0_4b96_990c_e69e311ded73.slice/crio-b89a6311b1dd1c5af95c9a32c77a5705526687cd9f585f3681c22510d5ac5bcc WatchSource:0}: Error finding container b89a6311b1dd1c5af95c9a32c77a5705526687cd9f585f3681c22510d5ac5bcc: Status 404 returned error can't find the container with id b89a6311b1dd1c5af95c9a32c77a5705526687cd9f585f3681c22510d5ac5bcc Jul 23 19:07:53.334888 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.334815 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:53.354653 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.354628 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn"] Jul 23 19:07:53.356433 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:07:53.356409 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2217e5d1_03bb_444a_a812_fac50591dcee.slice/crio-e665819bbfcffbfe7a8d80be8dbc968526416d722eb579662b6ae63b53548f22 WatchSource:0}: Error finding container e665819bbfcffbfe7a8d80be8dbc968526416d722eb579662b6ae63b53548f22: Status 404 returned error can't find the container with id e665819bbfcffbfe7a8d80be8dbc968526416d722eb579662b6ae63b53548f22 Jul 23 19:07:53.452734 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.452691 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:53.740927 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.740830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" event={"ID":"2217e5d1-03bb-444a-a812-fac50591dcee","Type":"ContainerStarted","Data":"e665819bbfcffbfe7a8d80be8dbc968526416d722eb579662b6ae63b53548f22"} Jul 23 19:07:53.742021 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:53.741996 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" event={"ID":"014d8ce8-f7f0-4b96-990c-e69e311ded73","Type":"ContainerStarted","Data":"b89a6311b1dd1c5af95c9a32c77a5705526687cd9f585f3681c22510d5ac5bcc"} Jul 23 19:07:56.139758 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:56.139718 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:07:56.140147 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:56.139808 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:07:56.140147 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:56.139879 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:07:56.758886 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:56.758855 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" event={"ID":"014d8ce8-f7f0-4b96-990c-e69e311ded73","Type":"ContainerStarted","Data":"b33c7c59c977ae6ac4a1212ea9c565af871b8ae87dad286d255b75a464e8ba6a"} Jul 23 19:07:56.812549 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:56.812492 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" podStartSLOduration=1.981177363 podStartE2EDuration="4.812470208s" totalImagesPullingTime="2.831292845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:07:56.809745808 +0000 UTC m=+424.388062563" watchObservedRunningTime="2026-07-23 19:07:56.812470208 +0000 UTC m=+424.390786963" Jul 23 19:07:57.109740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.109645 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:57.114602 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.114574 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:57.473142 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.473057 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:57.627809 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.627767 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:57.716181 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.716149 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-779d54d68-kghst" Jul 23 19:07:57.763153 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.763113 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:57.763996 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.763978 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678rksmv" Jul 23 19:07:57.914856 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:57.914808 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:58.185488 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:58.185394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:59.247400 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:59.247362 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:59.360404 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:59.360377 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:59.505971 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:59.505922 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:07:59.684685 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:07:59.684645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:04.050278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:04.050245 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:08:04.050562 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:04.050313 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:08:04.797817 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:04.797779 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" event={"ID":"2217e5d1-03bb-444a-a812-fac50591dcee","Type":"ContainerStarted","Data":"14ca3b0870b057a25957a43d7f06dec6c4c0fe5b83b0e8bec92fdcfd0a3a2cec"} Jul 23 19:08:04.798013 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:04.797935 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:08:04.844995 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:04.844943 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" podStartSLOduration=2.153335292 podStartE2EDuration="12.844926819s" totalImagesPullingTime="10.691591527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:08:04.844325589 +0000 UTC m=+432.422642343" watchObservedRunningTime="2026-07-23 19:08:04.844926819 +0000 UTC m=+432.423243573" Jul 23 19:08:05.804096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:05.804065 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-lt8fn" Jul 23 19:08:12.603025 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:12.602993 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:12.706801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:12.706762 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:45.401907 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:45.401860 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:45.492329 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:45.492290 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:45.660355 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:45.660251 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:46.356125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:46.356089 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:46.557474 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:46.557433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:56.994495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:56.994457 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:57.070716 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:57.070678 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:57.144818 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:57.144784 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:57.729345 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:57.729311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:08:57.831303 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:08:57.831275 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:08.420745 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:08.420711 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:08.520945 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:08.520910 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:08.594698 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:08.594662 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:10.739308 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:10.739270 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:11.356857 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:11.356801 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:12.424299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:12.424265 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:12.530368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:12.530331 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:15.974100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:15.974064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:18.569656 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:18.569617 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:19.058556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:19.058528 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:20.227960 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:20.227923 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:22.555781 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:22.555734 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:24.119021 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:24.118983 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:25.574718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:25.574682 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:25.674716 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:25.674684 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:25.978962 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:25.978881 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:26.077751 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:26.077713 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:29.601091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:29.601039 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:29.715945 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:29.715910 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:30.908176 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:30.908143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:31.077220 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:31.077183 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:33.964365 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:33.964323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:34.052421 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:34.052387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:40.820661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:40.820622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:40.919601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:40.919566 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:44.638655 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:44.638619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:44.796657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:44.796623 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:44.991452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:44.991373 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:45.091216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:45.091176 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:47.593410 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:47.593328 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:47.698734 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:47.698686 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:52.999480 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:52.999442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:53.099392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:53.099347 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:56.065647 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:56.065601 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:56.128064 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:56.128026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:59.628814 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:59.628776 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:09:59.730417 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:09:59.730374 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:13.969246 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:13.969209 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms"] Jul 23 19:10:13.972405 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:13.972389 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:13.983187 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:10:13.983160 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" podUID="3276c746-37b7-40ff-b653-7ede69c57ae6" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:13.983475 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:10:13.983452 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-default-gateway-openshift-default-dockercfg-zl6qb\" is forbidden: User \"system:node:ip-10-0-139-123.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-123.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zl6qb\"" type="*v1.Secret" Jul 23 19:10:14.000662 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.000634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:14.006943 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.006910 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms"] Jul 23 19:10:14.097259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097259 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097348 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98swp\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-kube-api-access-98swp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3276c746-37b7-40ff-b653-7ede69c57ae6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097458 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.097477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.097475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198562 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198521 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198562 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198581 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198682 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98swp\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-kube-api-access-98swp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.198802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198777 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3276c746-37b7-40ff-b653-7ede69c57ae6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.198805 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.199033 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.199080 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199235 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.199154 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.199272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.199760 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.199733 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3276c746-37b7-40ff-b653-7ede69c57ae6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.201094 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.201076 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.201145 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.201096 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.208020 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.207995 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.208571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.208549 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98swp\" (UniqueName: \"kubernetes.io/projected/3276c746-37b7-40ff-b653-7ede69c57ae6-kube-api-access-98swp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-2s6ms\" (UID: \"3276c746-37b7-40ff-b653-7ede69c57ae6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:14.893803 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.893775 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zl6qb\"" Jul 23 19:10:14.902988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:14.902968 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:15.034550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.034524 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms"] Jul 23 19:10:15.037107 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:10:15.037082 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3276c746_37b7_40ff_b653_7ede69c57ae6.slice/crio-1baf61e81f8ee44e02178023f639a798fe4df845587df385502c98e402deb964 WatchSource:0}: Error finding container 1baf61e81f8ee44e02178023f639a798fe4df845587df385502c98e402deb964: Status 404 returned error can't find the container with id 1baf61e81f8ee44e02178023f639a798fe4df845587df385502c98e402deb964 Jul 23 19:10:15.039099 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.039068 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:10:15.039186 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.039130 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:10:15.039186 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.039156 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:10:15.276852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.276803 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:15.321136 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.321102 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" event={"ID":"3276c746-37b7-40ff-b653-7ede69c57ae6","Type":"ContainerStarted","Data":"f8b95e56fd6ba3724b78fea72982c75d38eea840bd4aac10f21668b63116f012"} Jul 23 19:10:15.321322 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.321157 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" event={"ID":"3276c746-37b7-40ff-b653-7ede69c57ae6","Type":"ContainerStarted","Data":"1baf61e81f8ee44e02178023f639a798fe4df845587df385502c98e402deb964"} Jul 23 19:10:15.342375 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.342321 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" podStartSLOduration=2.342303708 podStartE2EDuration="2.342303708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:10:15.340666351 +0000 UTC m=+562.918983107" watchObservedRunningTime="2026-07-23 19:10:15.342303708 +0000 UTC m=+562.920620463" Jul 23 19:10:15.903663 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:15.903627 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:16.908804 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:16.908769 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:17.329903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:17.329877 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:17.330857 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:17.330823 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:10:32.257802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:32.257761 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:32.377801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:32.377764 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:32.480691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:32.480648 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:33.145278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:33.145246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:33.246517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:33.246467 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:52.925931 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:52.925899 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:10:52.926879 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:52.926856 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:10:53.010325 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:53.010285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:53.106245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:53.106217 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:10:53.529244 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:10:53.529210 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:06.171680 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:06.171628 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:06.760030 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:06.759992 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:13.664769 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:13.664732 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:13.740501 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:13.740470 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:13.813680 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:13.813645 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:14.133749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:14.133722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:17.486280 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:17.486191 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:17.592441 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:17.592406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:38.532384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:38.532334 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:49.805305 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:49.805246 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:51.423983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:51.423939 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:52.743629 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:52.743591 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:52.900164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:52.900129 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:11:58.748809 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.748726 2576 generic.go:356] "Generic (PLEG): container finished" podID="3276c746-37b7-40ff-b653-7ede69c57ae6" containerID="f8b95e56fd6ba3724b78fea72982c75d38eea840bd4aac10f21668b63116f012" exitCode=0 Jul 23 19:11:58.749218 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.748800 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" event={"ID":"3276c746-37b7-40ff-b653-7ede69c57ae6","Type":"ContainerDied","Data":"f8b95e56fd6ba3724b78fea72982c75d38eea840bd4aac10f21668b63116f012"} Jul 23 19:11:58.749218 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.749195 2576 scope.go:117] "RemoveContainer" containerID="f8b95e56fd6ba3724b78fea72982c75d38eea840bd4aac10f21668b63116f012" Jul 23 19:11:58.750154 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.750120 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:11:58.750277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.750205 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:11:58.750277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:58.750256 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:11:59.757762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:59.757723 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" event={"ID":"3276c746-37b7-40ff-b653-7ede69c57ae6","Type":"ContainerStarted","Data":"c55131bc5ad6f03024b4df933c1a34bb426a6b8ddbd8272aea9cad3863276ef9"} Jul 23 19:11:59.904023 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:59.903981 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:11:59.904880 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:59.904830 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2s6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 23 19:11:59.904953 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:11:59.904910 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" podUID="3276c746-37b7-40ff-b653-7ede69c57ae6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 23 19:12:00.904001 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:00.903963 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2s6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 23 19:12:00.904405 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:00.904023 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" podUID="3276c746-37b7-40ff-b653-7ede69c57ae6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 23 19:12:01.167935 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:01.167825 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:01.267719 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:01.267685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:01.903667 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:01.903631 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2s6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 23 19:12:01.903860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:01.903682 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" podUID="3276c746-37b7-40ff-b653-7ede69c57ae6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 23 19:12:02.329778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:02.329746 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:12:02.589043 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:02.588962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:02.903750 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:02.903669 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-2s6ms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body= Jul 23 19:12:02.903750 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:02.903726 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" podUID="3276c746-37b7-40ff-b653-7ede69c57ae6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" Jul 23 19:12:03.935460 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:03.935432 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:12:03.936533 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:03.936514 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-2s6ms" Jul 23 19:12:17.205100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:17.205060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:18.761547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.761507 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4"] Jul 23 19:12:18.765301 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.765274 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.768711 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.768684 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 19:12:18.768871 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.768684 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 19:12:18.768871 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.768684 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 19:12:18.768871 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.768684 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g5frh\"" Jul 23 19:12:18.780668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.780635 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4"] Jul 23 19:12:18.800297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.800264 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:18.823444 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.823444 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.823678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823482 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.823678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2ww2\" (UniqueName: \"kubernetes.io/projected/f819db0f-816d-47db-b3ef-65ea0e3554b1-kube-api-access-r2ww2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.823678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823633 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f819db0f-816d-47db-b3ef-65ea0e3554b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.823828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.823690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925068 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925027 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2ww2\" (UniqueName: \"kubernetes.io/projected/f819db0f-816d-47db-b3ef-65ea0e3554b1-kube-api-access-r2ww2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f819db0f-816d-47db-b3ef-65ea0e3554b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925140 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925618 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925650 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.925830 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.925686 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.927861 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.927808 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f819db0f-816d-47db-b3ef-65ea0e3554b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.928040 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.928017 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f819db0f-816d-47db-b3ef-65ea0e3554b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:18.934371 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:18.934333 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2ww2\" (UniqueName: \"kubernetes.io/projected/f819db0f-816d-47db-b3ef-65ea0e3554b1-kube-api-access-r2ww2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4\" (UID: \"f819db0f-816d-47db-b3ef-65ea0e3554b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:19.075775 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:19.075673 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:19.227072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:19.227047 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4"] Jul 23 19:12:19.229610 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:12:19.229576 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf819db0f_816d_47db_b3ef_65ea0e3554b1.slice/crio-01352e5f86d7a65a75a80fb6681ff0f5fc37d5bffe09d614d29b1ed0e9bf2c75 WatchSource:0}: Error finding container 01352e5f86d7a65a75a80fb6681ff0f5fc37d5bffe09d614d29b1ed0e9bf2c75: Status 404 returned error can't find the container with id 01352e5f86d7a65a75a80fb6681ff0f5fc37d5bffe09d614d29b1ed0e9bf2c75 Jul 23 19:12:19.231080 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:19.231063 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 19:12:19.844534 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:19.844494 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" event={"ID":"f819db0f-816d-47db-b3ef-65ea0e3554b1","Type":"ContainerStarted","Data":"01352e5f86d7a65a75a80fb6681ff0f5fc37d5bffe09d614d29b1ed0e9bf2c75"} Jul 23 19:12:20.850433 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:20.850397 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" event={"ID":"f819db0f-816d-47db-b3ef-65ea0e3554b1","Type":"ContainerStarted","Data":"1c0e2331501a6f2fa109b967fdbede6cb149724054069774b88bd046117e3c21"} Jul 23 19:12:20.908550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:20.908494 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" podStartSLOduration=1.690390194 podStartE2EDuration="2.90847545s" totalImagesPullingTime="1.218085256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:12:20.896934042 +0000 UTC m=+688.475250798" watchObservedRunningTime="2026-07-23 19:12:20.90847545 +0000 UTC m=+688.486792205" Jul 23 19:12:29.075992 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:29.075960 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:29.076452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:29.076038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:29.088630 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:29.088607 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:29.900098 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:29.900070 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4" Jul 23 19:12:30.956539 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:30.956479 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:37.501972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:37.501929 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:43.211036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:43.211002 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:43.312857 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:43.312809 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:12:56.632075 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:12:56.631991 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:13:04.403992 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:13:04.403956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:13:18.521703 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:13:18.521664 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:13:20.820518 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:13:20.820484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:14:17.476200 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:14:17.476086 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:14:17.547856 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:14:17.547816 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:14:18.650205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:14:18.650169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:14:18.754896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:14:18.754861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:14:51.661806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:14:51.661769 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:15:00.159531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.159494 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc"] Jul 23 19:15:00.168294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.168264 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" Jul 23 19:15:00.173100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.173075 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\"" Jul 23 19:15:00.175957 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.175932 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc"] Jul 23 19:15:00.178555 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.178536 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" Jul 23 19:15:00.308849 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.308811 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc"] Jul 23 19:15:00.311533 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:15:00.311502 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b212f0_9dc9_484c_8c56_ef1739ebd2d6.slice/crio-6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3 WatchSource:0}: Error finding container 6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3: Status 404 returned error can't find the container with id 6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3 Jul 23 19:15:00.520649 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:00.520610 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" event={"ID":"c9b212f0-9dc9-484c-8c56-ef1739ebd2d6","Type":"ContainerStarted","Data":"6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3"} Jul 23 19:15:04.542270 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:04.542230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" event={"ID":"c9b212f0-9dc9-484c-8c56-ef1739ebd2d6","Type":"ContainerStarted","Data":"2178f45d3426b9bba9b4e4006183bf1260d5667b0235135fa475fdacfb00e70c"} Jul 23 19:15:04.558289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:04.558237 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" podStartSLOduration=1.140773691 podStartE2EDuration="4.55822062s" totalImagesPullingTime="3.417446929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:15:04.557638593 +0000 UTC m=+852.135955347" watchObservedRunningTime="2026-07-23 19:15:04.55822062 +0000 UTC m=+852.136537378" Jul 23 19:15:05.548290 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:05.548248 2576 generic.go:356] "Generic (PLEG): container finished" podID="c9b212f0-9dc9-484c-8c56-ef1739ebd2d6" containerID="2178f45d3426b9bba9b4e4006183bf1260d5667b0235135fa475fdacfb00e70c" exitCode=0 Jul 23 19:15:05.548977 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:05.548950 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" event={"ID":"c9b212f0-9dc9-484c-8c56-ef1739ebd2d6","Type":"ContainerDied","Data":"2178f45d3426b9bba9b4e4006183bf1260d5667b0235135fa475fdacfb00e70c"} Jul 23 19:15:06.683443 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:06.683416 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" Jul 23 19:15:07.559072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:07.559042 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" Jul 23 19:15:07.559237 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:07.559081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747235-wlrrc" event={"ID":"c9b212f0-9dc9-484c-8c56-ef1739ebd2d6","Type":"ContainerDied","Data":"6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3"} Jul 23 19:15:07.559237 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:07.559127 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f063a922ec02698cc686b3629456d9d776ba2d452919815af95045ee99585e3" Jul 23 19:15:52.954761 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:52.954680 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:15:52.956761 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:15:52.956733 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:16:22.957373 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:16:22.957338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:16:24.180783 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:16:24.180747 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:16:59.588615 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:16:59.588574 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:00.988508 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:00.988474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:05.628518 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:05.628484 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:05.729848 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:05.729812 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:05.832321 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:05.832289 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:05.998984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:05.998906 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:12.011232 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:12.011195 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:17:12.082449 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:17:12.082406 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:18:16.357330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:18:16.357296 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:18:25.113265 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:18:25.113230 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:19:06.513146 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:19:06.513110 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:19:06.576353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:19:06.576324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:20:52.984819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:20:52.984782 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:20:52.986882 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:20:52.986859 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:21:49.440729 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:21:49.440655 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:22:10.157715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:22:10.157684 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:24:54.076092 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:24:54.076010 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:25:06.000125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:25:06.000092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:25:22.947819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:25:22.947774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:25:24.475143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:25:24.475112 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:25:53.009927 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:25:53.009897 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:25:53.011863 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:25:53.011810 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:26:46.128087 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:26:46.128006 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:26:55.056622 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:26:55.056582 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:27:35.704560 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:27:35.704527 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:27:41.328185 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:27:41.328146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:27:41.424829 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:27:41.424787 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:28:35.357587 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:28:35.357552 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:03.206757 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:03.206716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:06.260506 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:06.260477 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:06.346589 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:06.346553 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:06.705424 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:06.705324 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:12.899819 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:12.899785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:12.959778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:12.959744 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:13.121517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.121489 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:13.404399 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.404370 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:29:13.404709 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.404695 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9b212f0-9dc9-484c-8c56-ef1739ebd2d6" containerName="cleanup" Jul 23 19:29:13.404761 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.404710 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9b212f0-9dc9-484c-8c56-ef1739ebd2d6" containerName="cleanup" Jul 23 19:29:13.404795 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.404762 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9b212f0-9dc9-484c-8c56-ef1739ebd2d6" containerName="cleanup" Jul 23 19:29:13.407875 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.407852 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.410091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.410069 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-de58cb5e-openshift-default-dockercfg-8rr6g\"" Jul 23 19:29:13.431005 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.430979 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:29:13.432739 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.432716 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:13.529402 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.529371 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:13.575259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575398 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575453 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.575632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.575474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42g5d\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.676810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676726 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.676810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.676810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676796 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676826 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676868 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676903 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677120 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.676992 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g5d\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677360 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.677160 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677360 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.677200 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677759 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.677725 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677759 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.677737 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.677927 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.677816 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.679381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.679366 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.679432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.679409 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.685142 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.685113 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.685299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.685283 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42g5d\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d\") pod \"e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.718264 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.718240 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:13.851637 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.851610 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:29:13.853354 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:29:13.853324 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda379efd2_39a3_4af5_9d1c_22797a23d779.slice/crio-17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c WatchSource:0}: Error finding container 17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c: Status 404 returned error can't find the container with id 17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c Jul 23 19:29:13.855467 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.855435 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:13.855547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.855500 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:13.855547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:13.855536 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:14.133088 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:14.133049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerStarted","Data":"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde"} Jul 23 19:29:14.133470 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:14.133118 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerStarted","Data":"17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c"} Jul 23 19:29:14.157265 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:14.157206 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podStartSLOduration=1.157184828 podStartE2EDuration="1.157184828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:29:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:29:14.155416016 +0000 UTC m=+1701.733732772" watchObservedRunningTime="2026-07-23 19:29:14.157184828 +0000 UTC m=+1701.735501584" Jul 23 19:29:14.719219 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:14.719188 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:14.723978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:14.723955 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:15.137528 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:15.137500 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:15.138522 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:15.138505 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:15.521817 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:15.521745 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:15.625094 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:15.625061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:15.724500 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:15.724463 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:16.168438 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:16.168401 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:16.290315 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:16.290281 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:16.387674 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:16.387643 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:30.686537 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:30.686498 2576 patch_prober.go:28] interesting pod/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": read tcp 10.132.0.2:47120->10.132.0.27:15021: read: connection reset by peer" start-of-body= Jul 23 19:29:30.687028 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:30.686584 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": read tcp 10.132.0.2:47120->10.132.0.27:15021: read: connection reset by peer" Jul 23 19:29:31.208084 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.208048 2576 generic.go:356] "Generic (PLEG): container finished" podID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerID="1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde" exitCode=0 Jul 23 19:29:31.208260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.208092 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerDied","Data":"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde"} Jul 23 19:29:31.208466 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.208453 2576 scope.go:117] "RemoveContainer" containerID="1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde" Jul 23 19:29:31.209456 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.209423 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:31.209556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.209498 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:31.209556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:31.209530 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:32.215160 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:32.215127 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerStarted","Data":"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f"} Jul 23 19:29:32.719012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:32.718969 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:32.719272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:32.719254 2576 patch_prober.go:28] interesting pod/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 23 19:29:32.719334 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:32.719292 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 23 19:29:33.719301 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:33.719261 2576 patch_prober.go:28] interesting pod/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 23 19:29:33.719688 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:33.719320 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 23 19:29:34.718806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:34.718768 2576 patch_prober.go:28] interesting pod/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 23 19:29:34.719019 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:34.718823 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 23 19:29:35.718678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:35.718640 2576 patch_prober.go:28] interesting pod/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 23 19:29:35.719091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:35.718699 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 23 19:29:36.721880 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:36.721849 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:36.722275 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:36.722115 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:36.722826 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:36.722807 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:29:51.468090 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:51.468052 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.135500 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.135464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.264117 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.264089 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:29:56.264313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.264292 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" containerID="cri-o://d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f" gracePeriod=30 Jul 23 19:29:56.286969 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.286944 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.516469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.516444 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.764421 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.764387 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.863143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.863067 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:56.981993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:56.981964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:57.306099 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.306070 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:29:57.308107 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.308092 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.310703 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.310681 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a0642743-gw-openshift-default-dockercfg-glwj7\"" Jul 23 19:29:57.321434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.321410 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:29:57.337749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.337725 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:57.447991 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.447962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448050 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448071 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjxjx\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448103 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448252 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.448436 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.448354 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.548805 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548775 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548873 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548898 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjxjx\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548933 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.548997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549337 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549360 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549564 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549502 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549630 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549562 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.549998 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.549966 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.551414 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.551382 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.551801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.551780 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.557072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.557019 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjxjx\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.557072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.557063 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token\") pod \"e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.597159 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.597134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:57.618205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.618185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:57.804311 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.804282 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:29:57.805124 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:29:57.805098 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d183866_319a_467a_b7f3_38610940f4f6.slice/crio-db97d95712760abf778405d2c4b87b07e204d586837b72e4dc859b117747e90b WatchSource:0}: Error finding container db97d95712760abf778405d2c4b87b07e204d586837b72e4dc859b117747e90b: Status 404 returned error can't find the container with id db97d95712760abf778405d2c4b87b07e204d586837b72e4dc859b117747e90b Jul 23 19:29:57.807338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.807276 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:57.807422 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.807338 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:57.807422 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:57.807364 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:29:58.327994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:58.327962 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerStarted","Data":"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee"} Jul 23 19:29:58.328361 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:58.328017 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerStarted","Data":"db97d95712760abf778405d2c4b87b07e204d586837b72e4dc859b117747e90b"} Jul 23 19:29:58.353698 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:58.353647 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podStartSLOduration=1.353630874 podStartE2EDuration="1.353630874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:29:58.353369113 +0000 UTC m=+1745.931685866" watchObservedRunningTime="2026-07-23 19:29:58.353630874 +0000 UTC m=+1745.931947628" Jul 23 19:29:58.618818 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:58.618736 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:58.623634 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:58.623609 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:59.333044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:59.333015 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:59.333879 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:59.333859 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:29:59.823260 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:59.823200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:29:59.934514 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:29:59.934453 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:00.043687 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.043647 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:00.133767 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.133693 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd"] Jul 23 19:30:00.136092 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.136076 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" Jul 23 19:30:00.138862 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.138826 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\"" Jul 23 19:30:00.147279 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.147256 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" Jul 23 19:30:00.148087 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.148070 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd"] Jul 23 19:30:00.283773 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.283748 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd"] Jul 23 19:30:00.284987 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:30:00.284961 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98865747_66df_4dcf_b1bf_a7d6d0b3abdb.slice/crio-c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024 WatchSource:0}: Error finding container c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024: Status 404 returned error can't find the container with id c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024 Jul 23 19:30:00.338402 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.338377 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" event={"ID":"98865747-66df-4dcf-b1bf-a7d6d0b3abdb","Type":"ContainerStarted","Data":"c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024"} Jul 23 19:30:00.528632 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.528604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:00.629403 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:00.629376 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:01.343916 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.343882 2576 generic.go:356] "Generic (PLEG): container finished" podID="98865747-66df-4dcf-b1bf-a7d6d0b3abdb" containerID="b10173372e7a0a78149c434377a96a2d258d3f7e6b2f51d257b665c3b8e6933e" exitCode=0 Jul 23 19:30:01.344297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.343967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" event={"ID":"98865747-66df-4dcf-b1bf-a7d6d0b3abdb","Type":"ContainerDied","Data":"b10173372e7a0a78149c434377a96a2d258d3f7e6b2f51d257b665c3b8e6933e"} Jul 23 19:30:01.511771 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.511748 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:30:01.585356 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585327 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42g5d\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585366 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585384 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585418 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585434 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585505 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585471 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585513 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585549 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585584 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token\") pod \"a379efd2-39a3-4af5-9d1c-22797a23d779\" (UID: \"a379efd2-39a3-4af5-9d1c-22797a23d779\") " Jul 23 19:30:01.585904 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585803 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:01.585904 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585886 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:01.586044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.586020 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:01.586114 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.585959 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:30:01.586594 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.586558 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data" (OuterVolumeSpecName: "istio-data") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:01.587790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.587769 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:01.587790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.587776 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d" (OuterVolumeSpecName: "kube-api-access-42g5d") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "kube-api-access-42g5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:30:01.587957 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.587821 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token" (OuterVolumeSpecName: "istio-token") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:30:01.587957 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.587920 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a379efd2-39a3-4af5-9d1c-22797a23d779" (UID: "a379efd2-39a3-4af5-9d1c-22797a23d779"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686416 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42g5d\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-kube-api-access-42g5d\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686441 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686450 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686468 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686478 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686492 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686487 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a379efd2-39a3-4af5-9d1c-22797a23d779-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686516 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686528 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a379efd2-39a3-4af5-9d1c-22797a23d779-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:01.686778 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:01.686539 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a379efd2-39a3-4af5-9d1c-22797a23d779-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:02.350420 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.350390 2576 generic.go:356] "Generic (PLEG): container finished" podID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerID="d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f" exitCode=0 Jul 23 19:30:02.350924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.350439 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" Jul 23 19:30:02.350924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.350483 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerDied","Data":"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f"} Jul 23 19:30:02.350924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.350553 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6" event={"ID":"a379efd2-39a3-4af5-9d1c-22797a23d779","Type":"ContainerDied","Data":"17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c"} Jul 23 19:30:02.350924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.350574 2576 scope.go:117] "RemoveContainer" containerID="d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f" Jul 23 19:30:02.359168 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:30:02.358639 2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda379efd2_39a3_4af5_9d1c_22797a23d779.slice/crio-17307612d1e52aff7de83d3405be4d9d486d5ed49b78942794a5382147ce285c\": RecentStats: unable to find data in memory cache]" Jul 23 19:30:02.360424 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.360398 2576 scope.go:117] "RemoveContainer" containerID="1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde" Jul 23 19:30:02.368557 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.368538 2576 scope.go:117] "RemoveContainer" containerID="d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f" Jul 23 19:30:02.368870 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:30:02.368827 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f\": container with ID starting with d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f not found: ID does not exist" containerID="d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f" Jul 23 19:30:02.368966 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.368877 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f"} err="failed to get container status \"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f\": rpc error: code = NotFound desc = could not find container \"d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f\": container with ID starting with d4515425c35be14e67669b74f853a94066d2ffd63ccc57400c31793f90022a4f not found: ID does not exist" Jul 23 19:30:02.368966 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.368894 2576 scope.go:117] "RemoveContainer" containerID="1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde" Jul 23 19:30:02.369146 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:30:02.369126 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde\": container with ID starting with 1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde not found: ID does not exist" containerID="1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde" Jul 23 19:30:02.369238 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.369155 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde"} err="failed to get container status \"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde\": rpc error: code = NotFound desc = could not find container \"1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde\": container with ID starting with 1b11af35b499b84529a34d8aed7f8d8754e120815efa393ec08f751f96985dde not found: ID does not exist" Jul 23 19:30:02.375390 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.375361 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:30:02.378987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.378963 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-de58cb5e-openshift-default-8447bb8fb5-6scb6"] Jul 23 19:30:02.483706 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:02.483673 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" Jul 23 19:30:03.047135 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:03.047105 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" path="/var/lib/kubelet/pods/a379efd2-39a3-4af5-9d1c-22797a23d779/volumes" Jul 23 19:30:03.356450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:03.356373 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" Jul 23 19:30:03.356450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:03.356391 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747250-55ztd" event={"ID":"98865747-66df-4dcf-b1bf-a7d6d0b3abdb","Type":"ContainerDied","Data":"c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024"} Jul 23 19:30:03.356450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:03.356430 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c809e101acbb20bf7c65f2c4f9d5d309d3b7596d7e033ffc3b3905f9f97a4024" Jul 23 19:30:13.403633 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.403598 2576 generic.go:356] "Generic (PLEG): container finished" podID="6d183866-319a-467a-b7f3-38610940f4f6" containerID="281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee" exitCode=0 Jul 23 19:30:13.404078 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.403671 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerDied","Data":"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee"} Jul 23 19:30:13.404078 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.404043 2576 scope.go:117] "RemoveContainer" containerID="281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee" Jul 23 19:30:13.405043 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.405019 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:13.405136 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.405081 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:13.405136 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:13.405118 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:14.334010 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:14.333942 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:14.410016 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:14.409986 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerStarted","Data":"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4"} Jul 23 19:30:14.619273 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:14.619186 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:14.620427 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:14.620399 2576 patch_prober.go:28] interesting pod/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 23 19:30:14.620559 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:14.620461 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 23 19:30:15.618592 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:15.618559 2576 patch_prober.go:28] interesting pod/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 23 19:30:15.619004 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:15.618613 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 23 19:30:16.618743 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:16.618706 2576 patch_prober.go:28] interesting pod/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 23 19:30:16.619128 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:16.618762 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 23 19:30:17.619286 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:17.619248 2576 patch_prober.go:28] interesting pod/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" start-of-body= Jul 23 19:30:17.619794 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:17.619302 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.28:15021/healthz/ready\": dial tcp 10.132.0.28:15021: connect: connection refused" Jul 23 19:30:18.622210 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:18.622179 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:18.622615 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:18.622438 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:18.623129 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:18.623111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:35.408177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:35.408147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:39.948103 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:39.948070 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:40.089747 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.089713 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:30:40.090024 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.090000 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" containerID="cri-o://c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4" gracePeriod=30 Jul 23 19:30:40.192651 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.192604 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:40.353368 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.353338 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:40.593543 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.593515 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:40.924932 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.924900 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:30:40.925206 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925193 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98865747-66df-4dcf-b1bf-a7d6d0b3abdb" containerName="cleanup" Jul 23 19:30:40.925206 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925207 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="98865747-66df-4dcf-b1bf-a7d6d0b3abdb" containerName="cleanup" Jul 23 19:30:40.925293 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925222 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:30:40.925293 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925227 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:30:40.925293 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925235 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:30:40.925293 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925241 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:30:40.925413 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925297 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="98865747-66df-4dcf-b1bf-a7d6d0b3abdb" containerName="cleanup" Jul 23 19:30:40.925413 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.925306 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:30:40.928160 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.928144 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:40.930900 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.930877 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1705afd4-openshift-default-dockercfg-5hmcr\"" Jul 23 19:30:40.941600 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.941577 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:30:40.982299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:40.982250 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:41.027522 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027479 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.027719 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027532 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.027719 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.027820 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027717 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.027820 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.027820 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027776 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z8hb\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.028009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.028009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027907 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.028009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.027941 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129300 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129363 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129393 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129469 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129511 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z8hb\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129937 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129701 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.129937 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.129744 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.130030 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.130004 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.130284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.130264 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.130284 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.130275 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.131827 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.131807 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.132094 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.132075 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.140469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.140444 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z8hb\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.140807 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.140791 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token\") pod \"e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.238112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.238040 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:41.372146 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.372119 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:30:41.373749 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:30:41.373724 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9e6861_3f74_48c9_8b3c_544c6055f110.slice/crio-6e2776d0d03e6351ce36ef944814b3923064ee9b0f3cf9f62b3ae9247d0d20fe WatchSource:0}: Error finding container 6e2776d0d03e6351ce36ef944814b3923064ee9b0f3cf9f62b3ae9247d0d20fe: Status 404 returned error can't find the container with id 6e2776d0d03e6351ce36ef944814b3923064ee9b0f3cf9f62b3ae9247d0d20fe Jul 23 19:30:41.375872 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.375830 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:41.375938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.375903 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:41.375938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.375931 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:41.532243 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.532198 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerStarted","Data":"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9"} Jul 23 19:30:41.532430 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.532275 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerStarted","Data":"6e2776d0d03e6351ce36ef944814b3923064ee9b0f3cf9f62b3ae9247d0d20fe"} Jul 23 19:30:41.564867 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.564822 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:41.581239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:41.581190 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podStartSLOduration=1.581173954 podStartE2EDuration="1.581173954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:30:41.578379316 +0000 UTC m=+1789.156696075" watchObservedRunningTime="2026-07-23 19:30:41.581173954 +0000 UTC m=+1789.159490707" Jul 23 19:30:42.238347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:42.238315 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:42.243073 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:42.243050 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:42.537246 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:42.537218 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:42.538025 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:42.538010 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:43.062268 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:43.062235 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:43.159457 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:43.159426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:43.281943 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:43.281913 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:43.556907 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:43.556877 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:44.107432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:44.107405 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:44.227022 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:44.226988 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:30:45.337502 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.337479 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:45.370854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.370769 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.370854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.370798 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.370854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.370826 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371069 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.370966 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371069 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371026 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371069 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371052 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371083 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371162 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjxjx\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371179 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:30:45.371216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371188 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket\") pod \"6d183866-319a-467a-b7f3-38610940f4f6\" (UID: \"6d183866-319a-467a-b7f3-38610940f4f6\") " Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371459 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371484 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371547 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371666 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371684 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.371714 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371709 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.372048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371722 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d183866-319a-467a-b7f3-38610940f4f6-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.372048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.371993 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:45.373276 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.373252 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:30:45.373460 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.373434 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:30:45.373568 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.373544 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:30:45.373642 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.373625 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx" (OuterVolumeSpecName: "kube-api-access-gjxjx") pod "6d183866-319a-467a-b7f3-38610940f4f6" (UID: "6d183866-319a-467a-b7f3-38610940f4f6"). InnerVolumeSpecName "kube-api-access-gjxjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:30:45.472657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.472625 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjxjx\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-kube-api-access-gjxjx\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.472657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.472651 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d183866-319a-467a-b7f3-38610940f4f6-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.472657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.472660 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d183866-319a-467a-b7f3-38610940f4f6-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.472900 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.472668 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.472900 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.472676 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d183866-319a-467a-b7f3-38610940f4f6-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:30:45.550972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.550905 2576 generic.go:356] "Generic (PLEG): container finished" podID="6d183866-319a-467a-b7f3-38610940f4f6" containerID="c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4" exitCode=0 Jul 23 19:30:45.550972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.550947 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerDied","Data":"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4"} Jul 23 19:30:45.550972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.550954 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" Jul 23 19:30:45.551208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.550991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s" event={"ID":"6d183866-319a-467a-b7f3-38610940f4f6","Type":"ContainerDied","Data":"db97d95712760abf778405d2c4b87b07e204d586837b72e4dc859b117747e90b"} Jul 23 19:30:45.551208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.551012 2576 scope.go:117] "RemoveContainer" containerID="c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4" Jul 23 19:30:45.559579 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.559563 2576 scope.go:117] "RemoveContainer" containerID="281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee" Jul 23 19:30:45.566957 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.566939 2576 scope.go:117] "RemoveContainer" containerID="c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4" Jul 23 19:30:45.567194 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:30:45.567170 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4\": container with ID starting with c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4 not found: ID does not exist" containerID="c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4" Jul 23 19:30:45.567233 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.567207 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4"} err="failed to get container status \"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4\": rpc error: code = NotFound desc = could not find container \"c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4\": container with ID starting with c4cf193d928db0ad48aaec1f54a904abba66097f55dbc47b39f4d01e13abbfa4 not found: ID does not exist" Jul 23 19:30:45.567233 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.567223 2576 scope.go:117] "RemoveContainer" containerID="281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee" Jul 23 19:30:45.567412 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:30:45.567394 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee\": container with ID starting with 281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee not found: ID does not exist" containerID="281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee" Jul 23 19:30:45.567465 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.567418 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee"} err="failed to get container status \"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee\": rpc error: code = NotFound desc = could not find container \"281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee\": container with ID starting with 281d16732df026febb81659fe365f7eea87674660b3e356a5971e8d67ca476ee not found: ID does not exist" Jul 23 19:30:45.585864 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.585829 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:30:45.590335 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:45.590314 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a0642743-gw-openshift-default-756dfcc5c9-5w79s"] Jul 23 19:30:47.046717 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:47.046681 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d183866-319a-467a-b7f3-38610940f4f6" path="/var/lib/kubelet/pods/6d183866-319a-467a-b7f3-38610940f4f6/volumes" Jul 23 19:30:53.044065 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:53.044030 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:30:53.061710 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:53.061679 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:30:57.538091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.538050 2576 patch_prober.go:28] interesting pod/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 23 19:30:57.538501 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.538119 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 23 19:30:57.606738 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.606710 2576 generic.go:356] "Generic (PLEG): container finished" podID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerID="622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9" exitCode=0 Jul 23 19:30:57.606903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.606792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerDied","Data":"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9"} Jul 23 19:30:57.607221 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.607206 2576 scope.go:117] "RemoveContainer" containerID="622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9" Jul 23 19:30:57.608225 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.608198 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:57.608298 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.608281 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:57.608349 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:57.608332 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:30:58.613073 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:58.613036 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerStarted","Data":"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d"} Jul 23 19:30:59.238689 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:59.238654 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:30:59.239083 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:59.239058 2576 patch_prober.go:28] interesting pod/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 23 19:30:59.239282 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:30:59.239258 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 23 19:31:00.238618 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:00.238579 2576 patch_prober.go:28] interesting pod/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 23 19:31:00.239240 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:00.239206 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 23 19:31:01.238913 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:01.238870 2576 patch_prober.go:28] interesting pod/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 23 19:31:01.239495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:01.239462 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 23 19:31:02.239488 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:02.239449 2576 patch_prober.go:28] interesting pod/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 23 19:31:02.240119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:02.240086 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 23 19:31:03.242565 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:03.242535 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:31:03.243049 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:03.242814 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:31:03.243565 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:03.243549 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:31:19.507988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:19.507949 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:23.994757 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:23.994718 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:25.153640 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.153605 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:31:25.154072 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.153819 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" containerID="cri-o://7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d" gracePeriod=30 Jul 23 19:31:25.168792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.168758 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:25.401618 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.401557 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:25.632860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.632817 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:25.923645 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923570 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:31:25.923906 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923893 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:31:25.923965 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923909 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:31:25.924009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923970 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:31:25.924009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923979 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:31:25.924009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.923986 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a379efd2-39a3-4af5-9d1c-22797a23d779" containerName="istio-proxy" Jul 23 19:31:25.925780 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.925758 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:25.928920 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.928895 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f36f30bf-openshift-default-dockercfg-6jzsc\"" Jul 23 19:31:25.940330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.940304 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:31:25.952477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:25.952443 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:26.021956 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.021923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022162 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.021979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022162 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022162 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022105 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghb76\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022162 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022212 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022239 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.022384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.022283 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123444 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123398 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghb76\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123615 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123642 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.123668 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123663 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124067 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.123730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124067 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.124022 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124204 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.124182 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124264 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.124223 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124348 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.124330 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.124412 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.124361 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.125931 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.125910 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.126271 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.126252 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.130931 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.130910 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:26.143369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.143339 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghb76\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.143742 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.143720 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token\") pod \"e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.236005 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.235907 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:26.387076 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.387051 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:31:26.389282 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:31:26.389255 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ccbd8f_d1ef_44d6_9a77_f062c0732fcc.slice/crio-28b46d9f9383cb415ddae2925a108898e5d498bc0074b3bcee63dfca32b0e1e3 WatchSource:0}: Error finding container 28b46d9f9383cb415ddae2925a108898e5d498bc0074b3bcee63dfca32b0e1e3: Status 404 returned error can't find the container with id 28b46d9f9383cb415ddae2925a108898e5d498bc0074b3bcee63dfca32b0e1e3 Jul 23 19:31:26.391633 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.391597 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:26.391716 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.391690 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:26.391754 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.391722 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:26.734287 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.734251 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerStarted","Data":"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87"} Jul 23 19:31:26.734489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.734300 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerStarted","Data":"28b46d9f9383cb415ddae2925a108898e5d498bc0074b3bcee63dfca32b0e1e3"} Jul 23 19:31:26.761854 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:26.761777 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podStartSLOduration=1.761760444 podStartE2EDuration="1.761760444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:31:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:31:26.759875479 +0000 UTC m=+1834.338192236" watchObservedRunningTime="2026-07-23 19:31:26.761760444 +0000 UTC m=+1834.340077199" Jul 23 19:31:27.236470 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:27.236437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:27.241257 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:27.241229 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:27.739921 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:27.739884 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:27.740830 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:27.740813 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:28.216561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:28.216516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:28.357402 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:28.357367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:28.467986 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:28.467861 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:28.630267 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:28.630208 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:29.154722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:29.154685 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:29.249960 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:29.249931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:30.395525 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.395497 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:31:30.457205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457119 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457158 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457199 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457222 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457272 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z8hb\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457305 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457350 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457414 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457609 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:31:30.457749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457660 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:31:30.457749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457673 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:31:30.457749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457711 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data\") pod \"7d9e6861-3f74-48c9-8b3c-544c6055f110\" (UID: \"7d9e6861-3f74-48c9-8b3c-544c6055f110\") " Jul 23 19:31:30.457995 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.457965 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:31:30.458717 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.458691 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:31:30.458806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.458770 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.458806 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.458795 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.458938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.458808 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d9e6861-3f74-48c9-8b3c-544c6055f110-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.458938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.458822 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.459556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.459527 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb" (OuterVolumeSpecName: "kube-api-access-7z8hb") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "kube-api-access-7z8hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:31:30.459556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.459548 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:31:30.459707 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.459640 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:31:30.459707 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.459688 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d9e6861-3f74-48c9-8b3c-544c6055f110" (UID: "7d9e6861-3f74-48c9-8b3c-544c6055f110"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:31:30.560096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.560063 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.560096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.560093 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.560096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.560102 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.560096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.560110 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d9e6861-3f74-48c9-8b3c-544c6055f110-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.560372 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.560119 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z8hb\" (UniqueName: \"kubernetes.io/projected/7d9e6861-3f74-48c9-8b3c-544c6055f110-kube-api-access-7z8hb\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:31:30.756507 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.756476 2576 generic.go:356] "Generic (PLEG): container finished" podID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerID="7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d" exitCode=0 Jul 23 19:31:30.756677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.756529 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" Jul 23 19:31:30.756677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.756556 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerDied","Data":"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d"} Jul 23 19:31:30.756677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.756635 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6" event={"ID":"7d9e6861-3f74-48c9-8b3c-544c6055f110","Type":"ContainerDied","Data":"6e2776d0d03e6351ce36ef944814b3923064ee9b0f3cf9f62b3ae9247d0d20fe"} Jul 23 19:31:30.756677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.756658 2576 scope.go:117] "RemoveContainer" containerID="7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d" Jul 23 19:31:30.766041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.766022 2576 scope.go:117] "RemoveContainer" containerID="622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9" Jul 23 19:31:30.774296 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.774274 2576 scope.go:117] "RemoveContainer" containerID="7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d" Jul 23 19:31:30.774587 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:31:30.774566 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d\": container with ID starting with 7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d not found: ID does not exist" containerID="7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d" Jul 23 19:31:30.774644 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.774597 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d"} err="failed to get container status \"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d\": rpc error: code = NotFound desc = could not find container \"7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d\": container with ID starting with 7333e716ef0aad7f9bdf26cf3eeb620a455d849c79d73237c79e49c8bec79a8d not found: ID does not exist" Jul 23 19:31:30.774644 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.774616 2576 scope.go:117] "RemoveContainer" containerID="622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9" Jul 23 19:31:30.774872 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:31:30.774829 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9\": container with ID starting with 622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9 not found: ID does not exist" containerID="622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9" Jul 23 19:31:30.774927 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.774883 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9"} err="failed to get container status \"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9\": rpc error: code = NotFound desc = could not find container \"622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9\": container with ID starting with 622e9e4afdc5b1821398291b7551d94d2347cd5432bb7244fbeace6da05e20b9 not found: ID does not exist" Jul 23 19:31:30.780887 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.780861 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:31:30.784616 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:30.784594 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1705afd4-openshift-default-69ddd84b99-rc7t6"] Jul 23 19:31:31.046984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:31.046906 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" path="/var/lib/kubelet/pods/7d9e6861-3f74-48c9-8b3c-544c6055f110/volumes" Jul 23 19:31:41.806527 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.806448 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerID="1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87" exitCode=0 Jul 23 19:31:41.806527 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.806499 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerDied","Data":"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87"} Jul 23 19:31:41.806984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.806908 2576 scope.go:117] "RemoveContainer" containerID="1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87" Jul 23 19:31:41.807820 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.807793 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:41.807897 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.807879 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:41.807934 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:41.807915 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:31:42.740749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:42.740707 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:42.817229 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:42.817193 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerStarted","Data":"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207"} Jul 23 19:31:43.238410 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:43.238372 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:43.238773 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:43.238749 2576 patch_prober.go:28] interesting pod/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 23 19:31:43.238971 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:43.238942 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 23 19:31:44.236933 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:44.236896 2576 patch_prober.go:28] interesting pod/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 23 19:31:44.237413 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:44.236966 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 23 19:31:45.237171 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:45.237125 2576 patch_prober.go:28] interesting pod/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 23 19:31:45.237646 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:45.237198 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 23 19:31:46.245296 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:46.245254 2576 patch_prober.go:28] interesting pod/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 23 19:31:46.245902 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:46.245868 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 23 19:31:46.582004 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:46.581954 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:46.673483 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:46.673450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:47.240297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:47.240266 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:47.240550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:47.240527 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:47.241272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:47.241253 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:31:49.902808 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:49.902775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:31:50.003509 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:31:50.003474 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:03.463965 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:03.463932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:08.081994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:08.081955 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:08.223746 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:08.223713 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:32:08.223987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:08.223958 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" containerID="cri-o://040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207" gracePeriod=30 Jul 23 19:32:08.247826 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:08.247774 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:08.510041 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:08.510013 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:09.013094 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:09.013061 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:09.114173 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:09.114146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:09.213363 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:09.213336 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:10.748989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:10.748956 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:13.472051 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.472030 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:32:13.638764 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638734 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.638972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638801 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.638972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638826 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.638972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638874 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.638972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638909 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.638972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638946 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.639239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.638981 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.639239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639016 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghb76\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.639239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639046 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket\") pod \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\" (UID: \"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc\") " Jul 23 19:32:13.639239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639164 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:32:13.639239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639212 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:32:13.639479 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639417 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.639479 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639436 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.639479 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639458 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:32:13.639648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.639620 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:32:13.640265 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.640226 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data" (OuterVolumeSpecName: "istio-data") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:32:13.641159 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.641135 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:32:13.641267 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.641135 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:32:13.641340 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.641324 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76" (OuterVolumeSpecName: "kube-api-access-ghb76") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "kube-api-access-ghb76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:32:13.641475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.641454 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token" (OuterVolumeSpecName: "istio-token") pod "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" (UID: "67ccbd8f-d1ef-44d6-9a77-f062c0732fcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:32:13.740058 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740020 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740058 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740055 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740058 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740064 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740058 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740073 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740283 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740081 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740283 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740089 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghb76\" (UniqueName: \"kubernetes.io/projected/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-kube-api-access-ghb76\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.740283 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.740099 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:32:13.951191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.951106 2576 generic.go:356] "Generic (PLEG): container finished" podID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerID="040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207" exitCode=0 Jul 23 19:32:13.951191 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.951156 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" Jul 23 19:32:13.951369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.951185 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerDied","Data":"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207"} Jul 23 19:32:13.951369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.951242 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p" event={"ID":"67ccbd8f-d1ef-44d6-9a77-f062c0732fcc","Type":"ContainerDied","Data":"28b46d9f9383cb415ddae2925a108898e5d498bc0074b3bcee63dfca32b0e1e3"} Jul 23 19:32:13.951369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.951273 2576 scope.go:117] "RemoveContainer" containerID="040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207" Jul 23 19:32:13.960326 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.960306 2576 scope.go:117] "RemoveContainer" containerID="1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87" Jul 23 19:32:13.967649 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.967634 2576 scope.go:117] "RemoveContainer" containerID="040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207" Jul 23 19:32:13.967933 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:32:13.967912 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207\": container with ID starting with 040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207 not found: ID does not exist" containerID="040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207" Jul 23 19:32:13.967993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.967942 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207"} err="failed to get container status \"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207\": rpc error: code = NotFound desc = could not find container \"040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207\": container with ID starting with 040c4f139b3b4f37a0f0e4c03c02ca13dcfffbdf41d11247fbd62e04926b4207 not found: ID does not exist" Jul 23 19:32:13.967993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.967960 2576 scope.go:117] "RemoveContainer" containerID="1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87" Jul 23 19:32:13.968189 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:32:13.968174 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87\": container with ID starting with 1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87 not found: ID does not exist" containerID="1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87" Jul 23 19:32:13.968233 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.968194 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87"} err="failed to get container status \"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87\": rpc error: code = NotFound desc = could not find container \"1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87\": container with ID starting with 1e84cf85bbf0b867a7fb2f025f8c155c5fae82c1f7d9f22af38dd39c7eb25a87 not found: ID does not exist" Jul 23 19:32:13.971999 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.971980 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:32:13.976667 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:13.976647 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f36f30bf-openshift-default-5fcd95d6ff-fj45p"] Jul 23 19:32:15.046234 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:15.046204 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" path="/var/lib/kubelet/pods/67ccbd8f-d1ef-44d6-9a77-f062c0732fcc/volumes" Jul 23 19:32:16.756584 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:16.756518 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:16.861703 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:16.861673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:16.957490 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:16.957458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:17.057507 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:17.057425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:17.549186 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:17.549146 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:43.564789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:43.564739 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:50.246095 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:50.246060 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:50.358597 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:50.358545 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:32:50.450116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:32:50.450080 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:06.070595 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:06.070562 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:06.165964 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:06.165933 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:06.378607 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:06.378530 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:12.563988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:12.563957 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:12.669533 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:12.669500 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:12.768743 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:12.768715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:23.360731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:23.360699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:24.240025 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:24.239994 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:31.823111 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:31.823074 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:31.921101 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:31.921069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:32.021392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:32.021359 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:32.146402 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:32.146323 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:32.222503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:32.222470 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:33.015091 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:33.015015 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:33.109079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:33.109044 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:33.210004 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:33.209971 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:34.914762 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:34.914722 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:40.850593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:40.850560 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:40.910817 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:40.910785 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:41.549955 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:41.549921 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:47.832659 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:47.832577 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:47.933139 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:47.933098 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:48.088050 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.087962 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:48.394789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.394713 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:33:48.395074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395058 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395076 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395091 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395097 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395105 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395110 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d183866-319a-467a-b7f3-38610940f4f6" containerName="istio-proxy" Jul 23 19:33:48.395118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395118 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395123 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395185 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395192 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d9e6861-3f74-48c9-8b3c-544c6055f110" containerName="istio-proxy" Jul 23 19:33:48.395297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395199 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:33:48.395297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.395205 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:33:48.398239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.398221 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.400802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.400785 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4965db3e-openshift-default-dockercfg-zhwwd\"" Jul 23 19:33:48.410169 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.410150 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:48.421374 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.421349 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:33:48.511197 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.511169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:48.540123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540094 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540288 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540145 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540288 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540169 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540288 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540228 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540288 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540323 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540347 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.540432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.540373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmzz6\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641612 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641647 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641726 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641750 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.641785 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmzz6\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.641888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.642047 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.642095 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642330 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.642212 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.642441 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.642556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.642520 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.643973 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.643955 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.644204 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.644187 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.650573 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.650516 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.650815 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.650792 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmzz6\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6\") pod \"e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.708494 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.708464 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:48.842882 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.842852 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:33:48.843920 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:33:48.843894 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedcdbc1_63c5_442a_91ae_465476796873.slice/crio-12e4ddf92ec8cc5c04607637df4d24b5984d34274055c4865ceac82e844851b3 WatchSource:0}: Error finding container 12e4ddf92ec8cc5c04607637df4d24b5984d34274055c4865ceac82e844851b3: Status 404 returned error can't find the container with id 12e4ddf92ec8cc5c04607637df4d24b5984d34274055c4865ceac82e844851b3 Jul 23 19:33:48.846167 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.846137 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:33:48.846278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.846200 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:33:48.846278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:48.846230 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:33:49.357470 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.357430 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerStarted","Data":"bb947590761575e22eb435710eb7b3147da7e9f95a53d45a3465eb1241778112"} Jul 23 19:33:49.357661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.357500 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerStarted","Data":"12e4ddf92ec8cc5c04607637df4d24b5984d34274055c4865ceac82e844851b3"} Jul 23 19:33:49.378799 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.378741 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podStartSLOduration=1.378725543 podStartE2EDuration="1.378725543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:33:49.376259389 +0000 UTC m=+1976.954576145" watchObservedRunningTime="2026-07-23 19:33:49.378725543 +0000 UTC m=+1976.957042298" Jul 23 19:33:49.709688 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.709587 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:49.711189 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.711161 2576 patch_prober.go:28] interesting pod/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 23 19:33:49.711325 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:49.711214 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 23 19:33:50.713302 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:50.713271 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:51.317136 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.317104 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:51.367798 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.367771 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:51.368708 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.368687 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:33:51.394913 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.394882 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:51.574686 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.574611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:51.636148 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:51.636120 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:52.064063 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:52.064026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:52.165506 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:52.165469 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:55.831529 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:55.831496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:56.549430 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:56.549370 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:56.643495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:56.643462 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:33:57.015303 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:33:57.015273 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:03.073165 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.073131 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:03.174926 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.174891 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:03.423718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.423625 2576 generic.go:356] "Generic (PLEG): container finished" podID="dedcdbc1-63c5-442a-91ae-465476796873" containerID="bb947590761575e22eb435710eb7b3147da7e9f95a53d45a3465eb1241778112" exitCode=0 Jul 23 19:34:03.423718 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.423696 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerDied","Data":"bb947590761575e22eb435710eb7b3147da7e9f95a53d45a3465eb1241778112"} Jul 23 19:34:03.424143 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.424127 2576 scope.go:117] "RemoveContainer" containerID="bb947590761575e22eb435710eb7b3147da7e9f95a53d45a3465eb1241778112" Jul 23 19:34:03.425122 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.425092 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:03.425266 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.425156 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:03.425266 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:03.425184 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:04.429978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:04.429946 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerStarted","Data":"ca36f2efcc693820d8382e03c9d50265179af8b92f9fb090db0d53e44024909c"} Jul 23 19:34:04.709529 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:04.709439 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:04.710726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:04.710698 2576 patch_prober.go:28] interesting pod/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 23 19:34:04.710855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:04.710749 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 23 19:34:05.709048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:05.709009 2576 patch_prober.go:28] interesting pod/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 23 19:34:05.709520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:05.709103 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 23 19:34:06.367948 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:06.367901 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:06.709756 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:06.709666 2576 patch_prober.go:28] interesting pod/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 23 19:34:06.709756 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:06.709725 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 23 19:34:07.709133 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:07.709101 2576 patch_prober.go:28] interesting pod/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 23 19:34:07.709316 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:07.709166 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 23 19:34:08.712503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:08.712476 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:08.713641 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:08.713620 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:13.876795 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:13.876752 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:18.542243 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:18.542198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:18.920688 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:18.920593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.122739 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.122704 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.219940 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.219909 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.373289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.373209 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:34:25.373484 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.373452 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" containerID="cri-o://ca36f2efcc693820d8382e03c9d50265179af8b92f9fb090db0d53e44024909c" gracePeriod=30 Jul 23 19:34:25.403958 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.403932 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.621920 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.621884 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.759076 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.759043 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.850524 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.850496 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:25.951006 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:25.950972 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:30.553938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.553909 2576 generic.go:356] "Generic (PLEG): container finished" podID="dedcdbc1-63c5-442a-91ae-465476796873" containerID="ca36f2efcc693820d8382e03c9d50265179af8b92f9fb090db0d53e44024909c" exitCode=0 Jul 23 19:34:30.554396 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.553958 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerDied","Data":"ca36f2efcc693820d8382e03c9d50265179af8b92f9fb090db0d53e44024909c"} Jul 23 19:34:30.554396 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.554019 2576 scope.go:117] "RemoveContainer" containerID="bb947590761575e22eb435710eb7b3147da7e9f95a53d45a3465eb1241778112" Jul 23 19:34:30.623410 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.623387 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:30.723121 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723044 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723121 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723078 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723121 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723118 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723144 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723170 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723197 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723213 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723242 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmzz6\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723290 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo\") pod \"dedcdbc1-63c5-442a-91ae-465476796873\" (UID: \"dedcdbc1-63c5-442a-91ae-465476796873\") " Jul 23 19:34:30.723391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723353 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:34:30.723712 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723498 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:34:30.723798 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723771 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:34:30.723981 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.723951 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:34:30.724740 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.724713 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.724918 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.724900 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.725931 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.725895 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data" (OuterVolumeSpecName: "istio-data") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:34:30.731165 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.730654 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:34:30.731165 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.730820 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6" (OuterVolumeSpecName: "kube-api-access-mmzz6") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "kube-api-access-mmzz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:34:30.731369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.731234 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token" (OuterVolumeSpecName: "istio-token") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:34:30.731976 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.731953 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dedcdbc1-63c5-442a-91ae-465476796873" (UID: "dedcdbc1-63c5-442a-91ae-465476796873"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:34:30.826251 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826210 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmzz6\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-kube-api-access-mmzz6\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826251 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826242 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dedcdbc1-63c5-442a-91ae-465476796873-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826251 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826254 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dedcdbc1-63c5-442a-91ae-465476796873-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826263 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dedcdbc1-63c5-442a-91ae-465476796873-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826272 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826281 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:30.826495 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:30.826289 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dedcdbc1-63c5-442a-91ae-465476796873-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:34:31.559687 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.559658 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" Jul 23 19:34:31.560182 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.559663 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq" event={"ID":"dedcdbc1-63c5-442a-91ae-465476796873","Type":"ContainerDied","Data":"12e4ddf92ec8cc5c04607637df4d24b5984d34274055c4865ceac82e844851b3"} Jul 23 19:34:31.560182 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.559808 2576 scope.go:117] "RemoveContainer" containerID="ca36f2efcc693820d8382e03c9d50265179af8b92f9fb090db0d53e44024909c" Jul 23 19:34:31.576170 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.576145 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:34:31.578484 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.578456 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4965db3e-openshift-default-5bd584b65f-26dnq"] Jul 23 19:34:31.774665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.774628 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:31.881205 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.881132 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:31.974532 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:31.974502 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:32.140438 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:32.140356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:33.047007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:33.046974 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dedcdbc1-63c5-442a-91ae-465476796873" path="/var/lib/kubelet/pods/dedcdbc1-63c5-442a-91ae-465476796873/volumes" Jul 23 19:34:38.018082 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:38.018047 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:38.083889 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:38.083850 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:44.788437 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:44.788401 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:45.065919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.065826 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:34:45.066137 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066124 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:34:45.066187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066138 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:34:45.066187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066149 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:34:45.066187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066154 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ccbd8f-d1ef-44d6-9a77-f062c0732fcc" containerName="istio-proxy" Jul 23 19:34:45.066187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066164 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:34:45.066187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066170 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:34:45.066363 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.066242 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:34:45.070658 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.070636 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.073300 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.073272 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5ccd9e4a-openshift-default-dockercfg-4wqnk\"" Jul 23 19:34:45.084747 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.084702 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:34:45.088226 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.088199 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:45.146422 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146391 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146469 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z74v7\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146776 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146776 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146890 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146890 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.146967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.146889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.243060 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.243028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:45.248179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248157 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248195 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248278 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248245 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z74v7\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248401 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248620 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.248823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248795 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.249035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248818 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.249113 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248919 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.249113 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.248928 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.249113 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.249027 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.250660 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.250641 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.250751 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.250648 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.257119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.257095 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z74v7\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.257290 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.257272 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token\") pod \"e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.385081 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.385000 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:45.522280 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.522187 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:34:45.524523 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:34:45.524492 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae31e36_477e_4c8f_a485_adca10305bfb.slice/crio-528d6f028011a85d0e8ed4bb00657eff80d64fa9f3119ec973887a40616afd96 WatchSource:0}: Error finding container 528d6f028011a85d0e8ed4bb00657eff80d64fa9f3119ec973887a40616afd96: Status 404 returned error can't find the container with id 528d6f028011a85d0e8ed4bb00657eff80d64fa9f3119ec973887a40616afd96 Jul 23 19:34:45.526536 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.526499 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:45.526614 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.526579 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:45.526652 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.526619 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:34:45.620142 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.620114 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerStarted","Data":"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3"} Jul 23 19:34:45.620272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.620162 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerStarted","Data":"528d6f028011a85d0e8ed4bb00657eff80d64fa9f3119ec973887a40616afd96"} Jul 23 19:34:45.640749 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:45.640642 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podStartSLOduration=0.640623266 podStartE2EDuration="640.623266ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:34:45.639738752 +0000 UTC m=+2033.218055508" watchObservedRunningTime="2026-07-23 19:34:45.640623266 +0000 UTC m=+2033.218940022" Jul 23 19:34:46.386161 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:46.386126 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:46.390966 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:46.390934 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:46.624616 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:46.624586 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:46.625474 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:46.625456 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:34:47.565314 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:47.565278 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:47.645199 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:47.645169 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:47.714057 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:47.714028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:48.153427 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:48.153399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:48.252587 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:48.252554 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:48.360986 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:48.360938 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:52.087297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:52.087259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:34:57.578709 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:34:57.578676 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:00.694261 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.694160 2576 generic.go:356] "Generic (PLEG): container finished" podID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerID="8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3" exitCode=0 Jul 23 19:35:00.694261 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.694232 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerDied","Data":"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3"} Jul 23 19:35:00.694868 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.694739 2576 scope.go:117] "RemoveContainer" containerID="8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3" Jul 23 19:35:00.695863 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.695814 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:35:00.695936 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.695918 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:35:00.695990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:00.695976 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:35:01.624949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:01.624915 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:35:01.699924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:01.699892 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerStarted","Data":"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58"} Jul 23 19:35:02.385528 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:02.385492 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:35:02.386486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:02.386456 2576 patch_prober.go:28] interesting pod/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 19:35:02.386546 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:02.386528 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 19:35:03.385717 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:03.385675 2576 patch_prober.go:28] interesting pod/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 19:35:03.386225 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:03.385735 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 19:35:04.386012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:04.385977 2576 patch_prober.go:28] interesting pod/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 19:35:04.386506 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:04.386053 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 19:35:05.386229 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:05.386194 2576 patch_prober.go:28] interesting pod/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 23 19:35:05.386630 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:05.386249 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 23 19:35:06.389948 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:06.389920 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:35:06.390345 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:06.390201 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:35:06.390925 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:06.390904 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:35:06.527748 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:06.527715 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:25.472181 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.472098 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz"] Jul 23 19:35:25.472662 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.472461 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dedcdbc1-63c5-442a-91ae-465476796873" containerName="istio-proxy" Jul 23 19:35:25.475736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.475717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:25.478152 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.478132 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 19:35:25.478272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.478168 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\"" Jul 23 19:35:25.483557 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.483527 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz"] Jul 23 19:35:25.602204 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.602172 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz\" (UID: \"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:25.703692 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.703656 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz\" (UID: \"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:25.704244 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.704226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz\" (UID: \"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:25.787001 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.786976 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:25.920132 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:25.920071 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz"] Jul 23 19:35:25.921280 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:35:25.921256 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c57c7ae_ba9c_4b87_a2a4_5fe94a301e9e.slice/crio-f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50 WatchSource:0}: Error finding container f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50: Status 404 returned error can't find the container with id f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50 Jul 23 19:35:26.811644 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:26.811612 2576 generic.go:356] "Generic (PLEG): container finished" podID="3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" containerID="41593024c1ab337c9b097b06d2b72ce2b625e91daf171157232ea0e518506df7" exitCode=0 Jul 23 19:35:26.812006 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:26.811656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" event={"ID":"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e","Type":"ContainerDied","Data":"41593024c1ab337c9b097b06d2b72ce2b625e91daf171157232ea0e518506df7"} Jul 23 19:35:26.812006 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:26.811697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" event={"ID":"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e","Type":"ContainerStarted","Data":"f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50"} Jul 23 19:35:27.938306 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:27.938285 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:28.023127 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.023098 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca\") pod \"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e\" (UID: \"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e\") " Jul 23 19:35:28.023467 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.023442 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" (UID: "3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:35:28.123939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.123850 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e-maas-api-service-ca\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:35:28.821100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.821068 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" event={"ID":"3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e","Type":"ContainerDied","Data":"f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50"} Jul 23 19:35:28.821100 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.821095 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz" Jul 23 19:35:28.821312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:28.821115 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8711dfee1b19eb0d9e2004bd78e00e0ddd9930c0e1c488bda5f504f92793d50" Jul 23 19:35:41.010455 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:41.010419 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:45.553087 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:45.553049 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz"] Jul 23 19:35:45.555526 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:45.555499 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5ccd9e4a-sj4dz"] Jul 23 19:35:45.556290 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:45.556272 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:47.046518 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:47.046483 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" path="/var/lib/kubelet/pods/3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e/volumes" Jul 23 19:35:47.867270 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:47.867234 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:52.943422 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:52.943389 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:35:53.086724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:53.086699 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:35:53.090489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:35:53.090453 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:36:29.223566 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:29.223531 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:29.678450 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:29.678416 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:29.780502 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:29.780464 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:30.480456 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:30.480426 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:36.328213 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.328178 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:36.389926 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.389894 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:36.585112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.585035 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:36:36.585289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.585240 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" containerID="cri-o://e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58" gracePeriod=30 Jul 23 19:36:36.609452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.609421 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:36.845164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:36.845085 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:37.897761 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:37.897727 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:37.995109 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:37.995081 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:38.095013 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:38.094970 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:38.807642 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:38.807611 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:38.906520 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:38.906491 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:39.006415 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:39.006385 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:40.396998 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:40.396969 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:41.839370 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.839342 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:36:41.927190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927101 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927148 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z74v7\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927190 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927225 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927270 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927303 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927330 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927351 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927380 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927405 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo\") pod \"0ae31e36-477e-4c8f-a485-adca10305bfb\" (UID: \"0ae31e36-477e-4c8f-a485-adca10305bfb\") " Jul 23 19:36:41.927800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927542 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:36:41.927800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927614 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:36:41.927800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927671 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:41.927800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927689 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:41.927800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.927689 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:36:41.928427 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.928404 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:36:41.929447 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.929424 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:36:41.929589 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.929567 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:36:41.929802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.929776 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:36:41.929802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:41.929787 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7" (OuterVolumeSpecName: "kube-api-access-z74v7") pod "0ae31e36-477e-4c8f-a485-adca10305bfb" (UID: "0ae31e36-477e-4c8f-a485-adca10305bfb"). InnerVolumeSpecName "kube-api-access-z74v7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:36:42.028214 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028182 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ae31e36-477e-4c8f-a485-adca10305bfb-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028214 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028209 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028214 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028217 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028426 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028227 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028426 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028235 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z74v7\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-kube-api-access-z74v7\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028426 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028243 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.028426 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.028250 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ae31e36-477e-4c8f-a485-adca10305bfb-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:36:42.140571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.140540 2576 generic.go:356] "Generic (PLEG): container finished" podID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerID="e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58" exitCode=0 Jul 23 19:36:42.140731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.140584 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" Jul 23 19:36:42.140731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.140616 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerDied","Data":"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58"} Jul 23 19:36:42.140731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.140680 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq" event={"ID":"0ae31e36-477e-4c8f-a485-adca10305bfb","Type":"ContainerDied","Data":"528d6f028011a85d0e8ed4bb00657eff80d64fa9f3119ec973887a40616afd96"} Jul 23 19:36:42.140731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.140701 2576 scope.go:117] "RemoveContainer" containerID="e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58" Jul 23 19:36:42.149830 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.149815 2576 scope.go:117] "RemoveContainer" containerID="8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3" Jul 23 19:36:42.157390 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.157371 2576 scope.go:117] "RemoveContainer" containerID="e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58" Jul 23 19:36:42.157626 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:36:42.157609 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58\": container with ID starting with e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58 not found: ID does not exist" containerID="e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58" Jul 23 19:36:42.157670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.157634 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58"} err="failed to get container status \"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58\": rpc error: code = NotFound desc = could not find container \"e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58\": container with ID starting with e75af96bd2030602ff1e6bf471389ac075e44a108c89146b1ebfdd7f3b788a58 not found: ID does not exist" Jul 23 19:36:42.157670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.157651 2576 scope.go:117] "RemoveContainer" containerID="8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3" Jul 23 19:36:42.157892 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:36:42.157874 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3\": container with ID starting with 8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3 not found: ID does not exist" containerID="8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3" Jul 23 19:36:42.157939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.157897 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3"} err="failed to get container status \"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3\": rpc error: code = NotFound desc = could not find container \"8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3\": container with ID starting with 8fe1bcf396ee9a8b1fd06545a52b255d0a3a3f2297c7db7d06c31d58d19a2ee3 not found: ID does not exist" Jul 23 19:36:42.163746 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.163727 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:36:42.167791 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:42.167770 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5ccd9e4a-openshift-default-79bc8758c9-pwflq"] Jul 23 19:36:43.046331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:43.046302 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" path="/var/lib/kubelet/pods/0ae31e36-477e-4c8f-a485-adca10305bfb/volumes" Jul 23 19:36:46.866693 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:46.866608 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:46.967941 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:46.967907 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:47.376676 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:47.376639 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:47.473383 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:47.473351 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:47.574399 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:47.574367 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:53.845119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:53.845086 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:53.946983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:53.946950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:36:54.045475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:36:54.045445 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:04.565478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:04.565441 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:04.915510 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:04.915423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:25.879164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:25.879129 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:32.595169 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:32.595134 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:32.696606 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:32.696571 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:32.955948 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:32.955871 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:33.229623 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229542 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:37:33.229855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229829 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:33.229905 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229856 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:33.229905 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229873 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" containerName="revoke-keys" Jul 23 19:37:33.229905 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229879 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" containerName="revoke-keys" Jul 23 19:37:33.230007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229931 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:33.230007 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.229940 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c57c7ae-ba9c-4b87-a2a4-5fe94a301e9e" containerName="revoke-keys" Jul 23 19:37:33.232751 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.232734 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.235022 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.235001 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-453525-openshift-default-dockercfg-wtv8d\"" Jul 23 19:37:33.244557 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.244537 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:37:33.253070 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.253045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:33.342035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.341999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342035 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h89bj\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342158 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342190 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342263 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342205 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.342462 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.342264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443195 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443237 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443349 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h89bj\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443381 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443377 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443420 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443635 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443958 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443805 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.443958 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.443828 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.444083 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.444062 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.444295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.444276 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.445691 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.445662 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.445788 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.445777 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.454409 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.454383 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.454503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.454407 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h89bj\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj\") pod \"e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.543016 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.542985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:33.675063 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.675036 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:37:33.677089 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:37:33.677066 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1f2fb0_ac5e_4055_b372_322b92f13125.slice/crio-45baf3e2f113cfa2e42e18fc67afe6de0c07910f650143a0564e4fd50a855878 WatchSource:0}: Error finding container 45baf3e2f113cfa2e42e18fc67afe6de0c07910f650143a0564e4fd50a855878: Status 404 returned error can't find the container with id 45baf3e2f113cfa2e42e18fc67afe6de0c07910f650143a0564e4fd50a855878 Jul 23 19:37:33.679479 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.679441 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:33.679580 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.679508 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:33.679580 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.679537 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:33.730430 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:33.728678 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:34.367122 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:34.367086 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerStarted","Data":"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6"} Jul 23 19:37:34.367299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:34.367184 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerStarted","Data":"45baf3e2f113cfa2e42e18fc67afe6de0c07910f650143a0564e4fd50a855878"} Jul 23 19:37:34.388266 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:34.388219 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podStartSLOduration=1.388205073 podStartE2EDuration="1.388205073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:37:34.386827156 +0000 UTC m=+2201.965143903" watchObservedRunningTime="2026-07-23 19:37:34.388205073 +0000 UTC m=+2201.966521827" Jul 23 19:37:34.543577 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:34.543544 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:34.548171 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:34.548145 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:35.371753 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:35.371704 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:35.372517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:35.372501 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:36.328421 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:36.328361 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:36.416868 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:36.416826 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:36.558500 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:36.558442 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:36.778557 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:36.778516 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:37.186187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:37.186086 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:37.283042 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:37.283008 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:40.825993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:40.825960 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:41.142158 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.142069 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:41.423166 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.423094 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:37:41.423412 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.423399 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:41.423453 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.423413 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:41.423494 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.423472 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae31e36-477e-4c8f-a485-adca10305bfb" containerName="istio-proxy" Jul 23 19:37:41.426592 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.426573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.429348 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.429331 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d99fcd-openshift-default-dockercfg-5cqpw\"" Jul 23 19:37:41.441541 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.441523 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:37:41.449638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.449607 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:41.513828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.513794 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514018 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.513858 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514018 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.513886 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514018 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.513924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514018 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.513990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514203 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.514019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t2wn\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514203 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.514102 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514203 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.514133 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.514203 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.514154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.554351 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.554312 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:41.615628 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615596 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.615858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.615858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615701 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.615858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615751 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.615858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615793 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.615858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t2wn\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.616164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615939 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.616164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615963 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.616164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.615988 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.616315 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.616154 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.616366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.616322 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.617029 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.616986 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.617029 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.617001 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.618039 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.618017 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.619124 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.619102 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.619609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.619585 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.624446 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.624402 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.625125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.625103 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t2wn\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn\") pod \"e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.736765 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.736674 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:41.888055 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.888028 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:37:41.890737 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:37:41.890706 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1ba958_812f_4651_89a2_d17e14defe09.slice/crio-cfc41c4f4ec1ced9ea910f3e68af4573702d1cb00c543128d4d262a0114f9990 WatchSource:0}: Error finding container cfc41c4f4ec1ced9ea910f3e68af4573702d1cb00c543128d4d262a0114f9990: Status 404 returned error can't find the container with id cfc41c4f4ec1ced9ea910f3e68af4573702d1cb00c543128d4d262a0114f9990 Jul 23 19:37:41.895478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.895445 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:41.895563 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.895517 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:41.895563 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:41.895547 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:42.404069 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:42.404031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerStarted","Data":"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe"} Jul 23 19:37:42.404227 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:42.404097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerStarted","Data":"cfc41c4f4ec1ced9ea910f3e68af4573702d1cb00c543128d4d262a0114f9990"} Jul 23 19:37:42.424919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:42.424865 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podStartSLOduration=1.424849835 podStartE2EDuration="1.424849835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:37:42.422859673 +0000 UTC m=+2210.001176422" watchObservedRunningTime="2026-07-23 19:37:42.424849835 +0000 UTC m=+2210.003166581" Jul 23 19:37:42.737828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:42.737677 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:42.747043 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:42.747008 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:43.408666 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:43.408637 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:43.409532 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:43.409513 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:44.463293 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:44.463245 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:44.551873 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:44.551811 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:44.614590 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:44.614547 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:44.916630 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:44.916593 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:46.263299 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:46.263247 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:48.862177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:48.862147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:49.438379 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.438350 2576 generic.go:356] "Generic (PLEG): container finished" podID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerID="16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6" exitCode=0 Jul 23 19:37:49.438591 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.438398 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerDied","Data":"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6"} Jul 23 19:37:49.438789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.438775 2576 scope.go:117] "RemoveContainer" containerID="16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6" Jul 23 19:37:49.439773 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.439738 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:49.439866 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.439810 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:49.439866 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:49.439857 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:50.372482 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:50.372450 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:50.445268 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:50.445230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerStarted","Data":"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216"} Jul 23 19:37:50.544139 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:50.544095 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:50.545448 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:50.545409 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 19:37:50.545601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:50.545470 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 19:37:51.544065 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:51.544017 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 19:37:51.544623 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:51.544086 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 19:37:52.543750 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:52.543708 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 19:37:52.543978 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:52.543778 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 19:37:53.543628 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:53.543578 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 19:37:53.544196 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:53.543650 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 19:37:54.543794 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:54.543748 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 19:37:54.544338 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:54.543821 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 19:37:55.547984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:55.547953 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:55.548460 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:55.548338 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:55.549161 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:55.549140 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:37:58.410384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.410350 2576 patch_prober.go:28] interesting pod/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 19:37:58.410794 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.410415 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 19:37:58.515036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.515004 2576 generic.go:356] "Generic (PLEG): container finished" podID="0c1ba958-812f-4651-89a2-d17e14defe09" containerID="d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe" exitCode=0 Jul 23 19:37:58.515194 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.515055 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerDied","Data":"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe"} Jul 23 19:37:58.515470 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.515456 2576 scope.go:117] "RemoveContainer" containerID="d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe" Jul 23 19:37:58.516443 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.516417 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:58.516543 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.516484 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:58.516543 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:58.516517 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:37:59.205138 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.205106 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:59.522525 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.522493 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerStarted","Data":"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4"} Jul 23 19:37:59.736824 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.736789 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:37:59.737238 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.737208 2576 patch_prober.go:28] interesting pod/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 19:37:59.737454 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.737427 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 19:37:59.783674 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.783587 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:37:59.883741 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:37:59.883694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:00.737797 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:00.737755 2576 patch_prober.go:28] interesting pod/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 19:38:00.738439 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:00.738406 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 19:38:01.737870 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:01.737814 2576 patch_prober.go:28] interesting pod/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 19:38:01.738972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:01.738485 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 19:38:02.738042 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:02.737996 2576 patch_prober.go:28] interesting pod/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 23 19:38:02.738559 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:02.738072 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 23 19:38:03.767042 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:03.767008 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:38:03.767720 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:03.767689 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:38:03.769021 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:03.769001 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:38:05.821810 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:05.821773 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:05.920488 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:05.920458 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:08.493927 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:08.493896 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:18.695797 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:18.695719 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:51.188543 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:51.188507 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:58.124262 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:58.124229 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:59.037458 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.037431 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:59.153715 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.153683 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6"] Jul 23 19:38:59.156984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.156966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:38:59.159097 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.159069 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 19:38:59.159097 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.159093 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\"" Jul 23 19:38:59.164186 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.163929 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6"] Jul 23 19:38:59.230556 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.230519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-453525-rdwt6\" (UID: \"7c79aa4c-1eaf-455e-939b-13da4588d406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:38:59.331085 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.331000 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-453525-rdwt6\" (UID: \"7c79aa4c-1eaf-455e-939b-13da4588d406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:38:59.331621 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.331599 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-453525-rdwt6\" (UID: \"7c79aa4c-1eaf-455e-939b-13da4588d406\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:38:59.468224 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.468173 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:38:59.760198 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.760162 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:38:59.801821 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.801799 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6"] Jul 23 19:38:59.804235 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:38:59.804204 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c79aa4c_1eaf_455e_939b_13da4588d406.slice/crio-ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275 WatchSource:0}: Error finding container ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275: Status 404 returned error can't find the container with id ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275 Jul 23 19:38:59.860891 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:38:59.860858 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:00.800008 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:00.799979 2576 generic.go:356] "Generic (PLEG): container finished" podID="7c79aa4c-1eaf-455e-939b-13da4588d406" containerID="679eb6b9a7c5b48ac21234f9337812546c1ad1b422f43df7124b8b65ad9fcc76" exitCode=0 Jul 23 19:39:00.800392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:00.800059 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" event={"ID":"7c79aa4c-1eaf-455e-939b-13da4588d406","Type":"ContainerDied","Data":"679eb6b9a7c5b48ac21234f9337812546c1ad1b422f43df7124b8b65ad9fcc76"} Jul 23 19:39:00.800392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:00.800104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" event={"ID":"7c79aa4c-1eaf-455e-939b-13da4588d406","Type":"ContainerStarted","Data":"ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275"} Jul 23 19:39:01.937272 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:01.937245 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:39:02.056364 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.056275 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca\") pod \"7c79aa4c-1eaf-455e-939b-13da4588d406\" (UID: \"7c79aa4c-1eaf-455e-939b-13da4588d406\") " Jul 23 19:39:02.056601 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.056580 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7c79aa4c-1eaf-455e-939b-13da4588d406" (UID: "7c79aa4c-1eaf-455e-939b-13da4588d406"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:39:02.157752 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.157707 2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c79aa4c-1eaf-455e-939b-13da4588d406-maas-api-service-ca\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:02.809789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.809755 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" event={"ID":"7c79aa4c-1eaf-455e-939b-13da4588d406","Type":"ContainerDied","Data":"ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275"} Jul 23 19:39:02.809789 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.809779 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6" Jul 23 19:39:02.810040 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:02.809811 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab84e80d00b4a80766adb0562e4a18d714ee617650fd38d0b2f8c48d29da6275" Jul 23 19:39:09.617828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:09.617777 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:10.612678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:10.612640 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:10.668739 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:10.668708 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:14.214118 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:14.214079 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:14.229555 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:14.229525 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6"] Jul 23 19:39:14.234906 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:14.234880 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-453525-rdwt6"] Jul 23 19:39:14.608074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:14.608045 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:15.047279 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:15.047249 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c79aa4c-1eaf-455e-939b-13da4588d406" path="/var/lib/kubelet/pods/7c79aa4c-1eaf-455e-939b-13da4588d406/volumes" Jul 23 19:39:20.679498 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:20.679460 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:20.744898 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:20.744862 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:20.930233 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:20.930142 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:39:20.930457 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:20.930426 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" containerID="cri-o://74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216" gracePeriod=30 Jul 23 19:39:20.944993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:20.944969 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:21.166994 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:21.166964 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:21.965105 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:21.965073 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:22.050576 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:22.050550 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:25.550974 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:25.550934 2576 patch_prober.go:28] interesting pod/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 19:39:25.551341 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:25.550996 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 19:39:26.201249 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.201186 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:39:26.362689 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362653 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362689 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362694 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362722 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362743 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362783 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362803 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362829 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362905 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h89bj\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.362967 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362926 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy\") pod \"9b1f2fb0-ac5e-4055-b372-322b92f13125\" (UID: \"9b1f2fb0-ac5e-4055-b372-322b92f13125\") " Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.362992 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363123 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363166 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363250 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363268 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b1f2fb0-ac5e-4055-b372-322b92f13125-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363281 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.363366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363296 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:26.363782 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.363749 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:26.365081 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.365046 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:39:26.365251 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.365110 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:39:26.365251 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.365168 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:26.365339 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.365294 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj" (OuterVolumeSpecName: "kube-api-access-h89bj") pod "9b1f2fb0-ac5e-4055-b372-322b92f13125" (UID: "9b1f2fb0-ac5e-4055-b372-322b92f13125"). InnerVolumeSpecName "kube-api-access-h89bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463913 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463938 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463947 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463959 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463968 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h89bj\" (UniqueName: \"kubernetes.io/projected/9b1f2fb0-ac5e-4055-b372-322b92f13125-kube-api-access-h89bj\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.463989 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.463976 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b1f2fb0-ac5e-4055-b372-322b92f13125-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:26.918779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.918745 2576 generic.go:356] "Generic (PLEG): container finished" podID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerID="74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216" exitCode=0 Jul 23 19:39:26.919184 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.918800 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" Jul 23 19:39:26.919184 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.918848 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerDied","Data":"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216"} Jul 23 19:39:26.919184 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.918914 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9" event={"ID":"9b1f2fb0-ac5e-4055-b372-322b92f13125","Type":"ContainerDied","Data":"45baf3e2f113cfa2e42e18fc67afe6de0c07910f650143a0564e4fd50a855878"} Jul 23 19:39:26.919184 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.918939 2576 scope.go:117] "RemoveContainer" containerID="74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216" Jul 23 19:39:26.927990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.927974 2576 scope.go:117] "RemoveContainer" containerID="16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6" Jul 23 19:39:26.935677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.935662 2576 scope.go:117] "RemoveContainer" containerID="74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216" Jul 23 19:39:26.935937 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:39:26.935918 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216\": container with ID starting with 74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216 not found: ID does not exist" containerID="74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216" Jul 23 19:39:26.935987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.935945 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216"} err="failed to get container status \"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216\": rpc error: code = NotFound desc = could not find container \"74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216\": container with ID starting with 74e4a61428c13e47b8c8f4fb7c70343b98efc3e689403e0a864c3995b83dc216 not found: ID does not exist" Jul 23 19:39:26.935987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.935963 2576 scope.go:117] "RemoveContainer" containerID="16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6" Jul 23 19:39:26.936170 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:39:26.936155 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6\": container with ID starting with 16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6 not found: ID does not exist" containerID="16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6" Jul 23 19:39:26.936222 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.936172 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6"} err="failed to get container status \"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6\": rpc error: code = NotFound desc = could not find container \"16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6\": container with ID starting with 16d0b27e43b4285c358d6aa813a7e1fec1e46e329c07efe2a540399a4e975aa6 not found: ID does not exist" Jul 23 19:39:26.940679 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.940658 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:39:26.944322 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:26.944297 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-453525-openshift-default-85dc8d977c-dbtc9"] Jul 23 19:39:27.047546 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:27.047513 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" path="/var/lib/kubelet/pods/9b1f2fb0-ac5e-4055-b372-322b92f13125/volumes" Jul 23 19:39:31.690489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:31.690452 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:36.349823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:36.349791 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:36.715755 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:36.715673 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:42.693919 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:42.693889 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:42.751657 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:42.751628 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:42.938877 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:42.938826 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:39:42.939108 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:42.939084 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" containerID="cri-o://f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4" gracePeriod=30 Jul 23 19:39:42.961204 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:42.961126 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:43.186398 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.186366 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:43.508507 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.508475 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:43.783404 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783328 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:39:43.783847 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783821 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c79aa4c-1eaf-455e-939b-13da4588d406" containerName="revoke-keys" Jul 23 19:39:43.783938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783854 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c79aa4c-1eaf-455e-939b-13da4588d406" containerName="revoke-keys" Jul 23 19:39:43.783938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783884 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:43.783938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783893 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:43.783938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783905 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:43.783938 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.783915 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:43.784207 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.784005 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:43.784207 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.784018 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c79aa4c-1eaf-455e-939b-13da4588d406" containerName="revoke-keys" Jul 23 19:39:43.787380 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.787360 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.789655 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.789636 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bd74ec-openshift-default-dockercfg-28v79\"" Jul 23 19:39:43.799339 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.799321 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:39:43.813710 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813831 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813830 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4vrm\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813935 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.813979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813962 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.814116 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.813990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.814159 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.814117 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:43.814159 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.814117 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.914913 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.914865 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.914913 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.914913 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.914936 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.914965 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.914993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4vrm\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915088 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915164 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915530 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915414 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915530 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915510 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915614 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915568 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915614 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915583 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.915849 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.915820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.917371 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.917348 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.917684 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.917664 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.924274 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.924251 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:43.924440 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:43.924416 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4vrm\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm\") pod \"e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:44.098099 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.097998 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:44.176661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.176622 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:44.234240 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.234216 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:39:44.236558 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:39:44.236527 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a48958_5dcd_48df_b5c8_aa6cff97453d.slice/crio-88b1039ce287ab4d1181a4c325caa8c1afd787331e9bc6abd112262f109eb82c WatchSource:0}: Error finding container 88b1039ce287ab4d1181a4c325caa8c1afd787331e9bc6abd112262f109eb82c: Status 404 returned error can't find the container with id 88b1039ce287ab4d1181a4c325caa8c1afd787331e9bc6abd112262f109eb82c Jul 23 19:39:44.238676 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.238635 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:44.238765 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.238725 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:44.238860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:44.238770 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:45.003049 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:45.003018 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerStarted","Data":"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861"} Jul 23 19:39:45.003434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:45.003067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerStarted","Data":"88b1039ce287ab4d1181a4c325caa8c1afd787331e9bc6abd112262f109eb82c"} Jul 23 19:39:45.027621 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:45.027576 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podStartSLOduration=2.027560624 podStartE2EDuration="2.027560624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:39:45.025026631 +0000 UTC m=+2332.603343387" watchObservedRunningTime="2026-07-23 19:39:45.027560624 +0000 UTC m=+2332.605877430" Jul 23 19:39:45.098593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:45.098557 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:45.103876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:45.103830 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:46.007724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:46.007650 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:46.008506 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:46.008487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:39:46.959892 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:46.959851 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:47.049801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:47.049768 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:47.146036 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:47.145986 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:47.555673 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:47.555634 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:47.716869 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:47.716821 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:47.817887 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:47.817792 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:48.191139 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.191111 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:39:48.250110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250080 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250128 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250163 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250188 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250223 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250274 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t2wn\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250312 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250307 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250354 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250417 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data\") pod \"0c1ba958-812f-4651-89a2-d17e14defe09\" (UID: \"0c1ba958-812f-4651-89a2-d17e14defe09\") " Jul 23 19:39:48.250638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250539 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:48.250790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250727 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.250790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250723 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:48.250995 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.250855 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:48.251050 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.251007 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:39:48.251296 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.251275 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:48.252602 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.252574 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:39:48.252708 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.252677 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn" (OuterVolumeSpecName: "kube-api-access-2t2wn") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "kube-api-access-2t2wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:39:48.252790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.252771 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:39:48.252983 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.252958 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c1ba958-812f-4651-89a2-d17e14defe09" (UID: "0c1ba958-812f-4651-89a2-d17e14defe09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:39:48.351741 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351712 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351741 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351737 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c1ba958-812f-4651-89a2-d17e14defe09-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351741 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351745 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351759 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c1ba958-812f-4651-89a2-d17e14defe09-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351767 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351776 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c1ba958-812f-4651-89a2-d17e14defe09-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351783 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:48.351990 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:48.351790 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t2wn\" (UniqueName: \"kubernetes.io/projected/0c1ba958-812f-4651-89a2-d17e14defe09-kube-api-access-2t2wn\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:39:49.022801 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.022765 2576 generic.go:356] "Generic (PLEG): container finished" podID="0c1ba958-812f-4651-89a2-d17e14defe09" containerID="f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4" exitCode=0 Jul 23 19:39:49.022984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.022860 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" Jul 23 19:39:49.022984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.022868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerDied","Data":"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4"} Jul 23 19:39:49.022984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.022961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g" event={"ID":"0c1ba958-812f-4651-89a2-d17e14defe09","Type":"ContainerDied","Data":"cfc41c4f4ec1ced9ea910f3e68af4573702d1cb00c543128d4d262a0114f9990"} Jul 23 19:39:49.022984 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.022984 2576 scope.go:117] "RemoveContainer" containerID="f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4" Jul 23 19:39:49.031877 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.031858 2576 scope.go:117] "RemoveContainer" containerID="d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe" Jul 23 19:39:49.040030 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.040014 2576 scope.go:117] "RemoveContainer" containerID="f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4" Jul 23 19:39:49.040292 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:39:49.040274 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4\": container with ID starting with f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4 not found: ID does not exist" containerID="f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4" Jul 23 19:39:49.040355 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.040300 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4"} err="failed to get container status \"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4\": rpc error: code = NotFound desc = could not find container \"f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4\": container with ID starting with f51cb6846e4417ec39f23f027894dad1370e1cdacd935b97f024982e526d9aa4 not found: ID does not exist" Jul 23 19:39:49.040355 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.040316 2576 scope.go:117] "RemoveContainer" containerID="d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe" Jul 23 19:39:49.040552 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:39:49.040534 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe\": container with ID starting with d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe not found: ID does not exist" containerID="d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe" Jul 23 19:39:49.040606 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.040558 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe"} err="failed to get container status \"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe\": rpc error: code = NotFound desc = could not find container \"d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe\": container with ID starting with d9ff7cd83e705b24d5233437762113862db563531cf7aaee7f0010fa234e3abe not found: ID does not exist" Jul 23 19:39:49.047340 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.047319 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:39:49.051717 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:49.051690 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d99fcd-openshift-default-858494cdc7-drb7g"] Jul 23 19:39:51.046750 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:51.046709 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" path="/var/lib/kubelet/pods/0c1ba958-812f-4651-89a2-d17e14defe09/volumes" Jul 23 19:39:51.462336 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:51.462259 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:51.907517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:51.907489 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:52.169470 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169380 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:39:52.169852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169693 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:39:52.169852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169704 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:39:52.169852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169758 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:39:52.169852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169767 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:39:52.169852 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.169775 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1f2fb0-ac5e-4055-b372-322b92f13125" containerName="istio-proxy" Jul 23 19:39:52.174137 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.174112 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.176524 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.176505 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7e6d6b-openshift-default-dockercfg-qjn5c\"" Jul 23 19:39:52.185123 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.185105 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:39:52.197189 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.197167 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:52.283941 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.283896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.283980 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284125 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284052 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fzlr\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284266 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284239 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284331 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284322 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284417 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.284545 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.284441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.323106 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.323075 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:52.385661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.385876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385673 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.385876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.385876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385717 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.385876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.385876 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.385787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fzlr\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386076 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386122 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386177 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386148 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386359 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386077 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386419 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386397 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386490 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386425 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.386645 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.386629 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.388012 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.387995 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.388141 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.388125 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.393663 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.393638 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.393885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.393865 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fzlr\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr\") pod \"e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.484988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.484915 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:52.622181 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.622139 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:39:52.624468 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:39:52.624441 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf421f22f_49cd_42b5_9949_2eabecde19d3.slice/crio-c3252b19b9bdd015ed445527684a03995df8c1c8bd3479449d1c4d025c8e8426 WatchSource:0}: Error finding container c3252b19b9bdd015ed445527684a03995df8c1c8bd3479449d1c4d025c8e8426: Status 404 returned error can't find the container with id c3252b19b9bdd015ed445527684a03995df8c1c8bd3479449d1c4d025c8e8426 Jul 23 19:39:52.626613 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.626577 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:52.626684 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.626662 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:52.626723 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:52.626699 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:39:53.048559 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:53.048523 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerStarted","Data":"5aa697904e6c318b3c3852fa534db294ed31be93bca9791a0e4f7c576eae26c8"} Jul 23 19:39:53.048802 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:53.048574 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerStarted","Data":"c3252b19b9bdd015ed445527684a03995df8c1c8bd3479449d1c4d025c8e8426"} Jul 23 19:39:53.067772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:53.067722 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podStartSLOduration=1.067708126 podStartE2EDuration="1.067708126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:39:53.066159075 +0000 UTC m=+2340.644475831" watchObservedRunningTime="2026-07-23 19:39:53.067708126 +0000 UTC m=+2340.646024881" Jul 23 19:39:53.486006 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:53.485916 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:53.490924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:53.490901 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:54.051456 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:54.051423 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:54.056400 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:54.056372 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:39:54.988947 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:54.988914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:55.089722 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:55.089690 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:55.164295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:55.164253 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:55.428895 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:55.428863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:55.770350 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:55.770320 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:55.870175 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:55.870145 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:39:59.482903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:39:59.482863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:01.009362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.009327 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 19:40:01.009780 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.009383 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 19:40:01.086744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.086714 2576 generic.go:356] "Generic (PLEG): container finished" podID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerID="4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861" exitCode=0 Jul 23 19:40:01.086914 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.086788 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerDied","Data":"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861"} Jul 23 19:40:01.087158 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.087145 2576 scope.go:117] "RemoveContainer" containerID="4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861" Jul 23 19:40:01.088160 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.088133 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:01.088231 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.088217 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:01.088268 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:01.088257 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:02.092709 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:02.092673 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerStarted","Data":"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600"} Jul 23 19:40:02.098349 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:02.098320 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:02.098522 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:02.098502 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 19:40:02.098574 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:02.098548 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 19:40:03.099062 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:03.099028 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 19:40:03.099469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:03.099090 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 19:40:04.098545 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:04.098511 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 19:40:04.098737 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:04.098564 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 19:40:05.098538 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:05.098497 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 23 19:40:05.099053 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:05.098571 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 23 19:40:06.102690 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:06.102659 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:06.103208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:06.103078 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:06.103600 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:06.103581 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:07.951232 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:07.951198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:08.050855 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:08.050796 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:13.143747 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.143714 2576 generic.go:356] "Generic (PLEG): container finished" podID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerID="5aa697904e6c318b3c3852fa534db294ed31be93bca9791a0e4f7c576eae26c8" exitCode=0 Jul 23 19:40:13.144176 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.143783 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerDied","Data":"5aa697904e6c318b3c3852fa534db294ed31be93bca9791a0e4f7c576eae26c8"} Jul 23 19:40:13.144176 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.144167 2576 scope.go:117] "RemoveContainer" containerID="5aa697904e6c318b3c3852fa534db294ed31be93bca9791a0e4f7c576eae26c8" Jul 23 19:40:13.145151 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.145124 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:13.145252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.145207 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:13.145313 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:13.145251 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:40:14.157180 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:14.157135 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerStarted","Data":"77cd1c1ab374ca7e1ec5e1dce91bc924d93e286a93a8487a8c23ce6b0921677d"} Jul 23 19:40:14.485433 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:14.485346 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:40:14.485607 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:14.485566 2576 patch_prober.go:28] interesting pod/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 19:40:14.485676 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:14.485620 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 19:40:15.485531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:15.485498 2576 patch_prober.go:28] interesting pod/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 19:40:15.485939 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:15.485555 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 19:40:16.486249 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:16.486211 2576 patch_prober.go:28] interesting pod/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 19:40:16.486699 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:16.486280 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 19:40:16.521665 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:16.521635 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:16.619670 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:16.619638 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:17.486044 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:17.486008 2576 patch_prober.go:28] interesting pod/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 23 19:40:17.486230 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:17.486064 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 23 19:40:18.489190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:18.489157 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:40:18.489602 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:18.489421 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:40:18.489991 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:18.489972 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:40:22.545109 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:22.545065 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:27.539003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:27.538966 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:27.906222 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:27.906147 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:33.938605 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:33.938575 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:33.999731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:33.999700 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:34.255723 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:34.255699 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:34.263384 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:34.263360 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:40:34.263564 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:34.263546 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" containerID="cri-o://0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600" gracePeriod=30 Jul 23 19:40:34.458946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:34.458914 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:36.105434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:36.105398 2576 patch_prober.go:28] interesting pod/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 19:40:36.105807 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:36.105461 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 19:40:39.500156 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.500130 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:39.597003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.596913 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.596952 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.596983 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597005 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597049 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4vrm\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597082 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597128 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597173 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597243 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket\") pod \"45a48958-5dcd-48df-b5c8-aa6cff97453d\" (UID: \"45a48958-5dcd-48df-b5c8-aa6cff97453d\") " Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597250 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:40:39.597346 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597289 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:40:39.597678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597552 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.597678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597573 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.597678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597575 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:40:39.597678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.597590 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:40:39.598048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.598020 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data" (OuterVolumeSpecName: "istio-data") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:40:39.599522 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.599496 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:40:39.599654 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.599611 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm" (OuterVolumeSpecName: "kube-api-access-j4vrm") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "kube-api-access-j4vrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:40:39.599734 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.599664 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:40:39.599734 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.599706 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token" (OuterVolumeSpecName: "istio-token") pod "45a48958-5dcd-48df-b5c8-aa6cff97453d" (UID: "45a48958-5dcd-48df-b5c8-aa6cff97453d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:40:39.698305 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698274 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698305 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698301 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45a48958-5dcd-48df-b5c8-aa6cff97453d-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698305 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698310 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4vrm\" (UniqueName: \"kubernetes.io/projected/45a48958-5dcd-48df-b5c8-aa6cff97453d-kube-api-access-j4vrm\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698320 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698329 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698336 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:39.698571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:39.698344 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45a48958-5dcd-48df-b5c8-aa6cff97453d-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:40:40.280277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.280242 2576 generic.go:356] "Generic (PLEG): container finished" podID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerID="0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600" exitCode=0 Jul 23 19:40:40.280478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.280290 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" Jul 23 19:40:40.280478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.280323 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerDied","Data":"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600"} Jul 23 19:40:40.280478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.280371 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m" event={"ID":"45a48958-5dcd-48df-b5c8-aa6cff97453d","Type":"ContainerDied","Data":"88b1039ce287ab4d1181a4c325caa8c1afd787331e9bc6abd112262f109eb82c"} Jul 23 19:40:40.280478 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.280392 2576 scope.go:117] "RemoveContainer" containerID="0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600" Jul 23 19:40:40.289531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.289515 2576 scope.go:117] "RemoveContainer" containerID="4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861" Jul 23 19:40:40.296781 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.296765 2576 scope.go:117] "RemoveContainer" containerID="0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600" Jul 23 19:40:40.297055 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:40:40.297035 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600\": container with ID starting with 0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600 not found: ID does not exist" containerID="0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600" Jul 23 19:40:40.297129 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.297062 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600"} err="failed to get container status \"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600\": rpc error: code = NotFound desc = could not find container \"0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600\": container with ID starting with 0fc92b9605f246a4e8d9722e3b2df7823b93fa70cf5c27db658e4d0e6732c600 not found: ID does not exist" Jul 23 19:40:40.297129 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.297079 2576 scope.go:117] "RemoveContainer" containerID="4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861" Jul 23 19:40:40.297327 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:40:40.297308 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861\": container with ID starting with 4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861 not found: ID does not exist" containerID="4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861" Jul 23 19:40:40.297389 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.297336 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861"} err="failed to get container status \"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861\": rpc error: code = NotFound desc = could not find container \"4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861\": container with ID starting with 4201e3167daa01332e4d03a0d13fc949ca22c02bce5a74f86bcccca096dcd861 not found: ID does not exist" Jul 23 19:40:40.301612 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.301592 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:40:40.305737 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:40.305713 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bd74ec-openshift-default-d6f645c69-mnl4m"] Jul 23 19:40:41.046547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:41.046503 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" path="/var/lib/kubelet/pods/45a48958-5dcd-48df-b5c8-aa6cff97453d/volumes" Jul 23 19:40:49.959308 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:49.959277 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:53.111609 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:53.111581 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:40:53.116369 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:53.116348 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:40:54.631238 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:54.631202 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:40:55.003504 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:40:55.003417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:00.886685 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:00.886650 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:00.948452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:00.948425 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:01.176792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:01.176713 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:41:01.177018 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:01.176990 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" containerID="cri-o://77cd1c1ab374ca7e1ec5e1dce91bc924d93e286a93a8487a8c23ce6b0921677d" gracePeriod=30 Jul 23 19:41:01.202948 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:01.202918 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:01.389788 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:01.389753 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:01.742728 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:01.742694 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:02.005726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.005695 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:41:02.006182 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006165 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:41:02.006252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006184 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1ba958-812f-4651-89a2-d17e14defe09" containerName="istio-proxy" Jul 23 19:41:02.006252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006205 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.006252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006214 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.006252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006229 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.006252 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006238 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.006499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006325 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.006499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.006338 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a48958-5dcd-48df-b5c8-aa6cff97453d" containerName="istio-proxy" Jul 23 19:41:02.009892 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.009864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.012052 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.012034 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d5ee7b-openshift-default-dockercfg-q6mr7\"" Jul 23 19:41:02.025227 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.025200 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:41:02.029048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.029026 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:02.075545 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075512 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075653 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075724 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075686 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4v49\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075824 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075888 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.075987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075929 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.076128 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.075992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177014 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177048 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4v49\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177108 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177175 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177394 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177411 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177517 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177754 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177872 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177798 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.177872 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.177820 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.178079 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.178059 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.179442 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.179421 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.179731 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.179712 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.184851 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.184815 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.185006 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.184988 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4v49\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49\") pod \"e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.325745 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.325670 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:02.461894 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.461868 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:41:02.463951 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:41:02.463924 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8940cb83_eb04_4713_bc28_7d71e680cb26.slice/crio-82d765c6edf27065f2f924fd38c3f77ea4e8cf8a036fd5758661b603c4a8dfcd WatchSource:0}: Error finding container 82d765c6edf27065f2f924fd38c3f77ea4e8cf8a036fd5758661b603c4a8dfcd: Status 404 returned error can't find the container with id 82d765c6edf27065f2f924fd38c3f77ea4e8cf8a036fd5758661b603c4a8dfcd Jul 23 19:41:02.466480 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.466440 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:02.466591 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.466529 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:02.466591 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.466557 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:02.755650 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:02.755619 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:03.386813 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:03.386782 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerStarted","Data":"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed"} Jul 23 19:41:03.387216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:03.386830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerStarted","Data":"82d765c6edf27065f2f924fd38c3f77ea4e8cf8a036fd5758661b603c4a8dfcd"} Jul 23 19:41:03.409587 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:03.409537 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podStartSLOduration=2.409522863 podStartE2EDuration="2.409522863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:03.407802205 +0000 UTC m=+2410.986118960" watchObservedRunningTime="2026-07-23 19:41:03.409522863 +0000 UTC m=+2410.987839616" Jul 23 19:41:03.491264 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:03.491232 2576 patch_prober.go:28] interesting pod/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 19:41:03.491414 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:03.491285 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 19:41:04.326686 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:04.326653 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:04.331485 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:04.331461 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:04.392425 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:04.392399 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:04.393133 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:04.393116 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:05.379172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:05.379143 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:05.451987 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:05.451943 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:05.694946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:05.694829 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:05.888491 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:05.888433 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:06.221231 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.221200 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:06.329739 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.329706 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:06.404503 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.404471 2576 generic.go:356] "Generic (PLEG): container finished" podID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerID="77cd1c1ab374ca7e1ec5e1dce91bc924d93e286a93a8487a8c23ce6b0921677d" exitCode=0 Jul 23 19:41:06.404703 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.404550 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerDied","Data":"77cd1c1ab374ca7e1ec5e1dce91bc924d93e286a93a8487a8c23ce6b0921677d"} Jul 23 19:41:06.404703 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.404641 2576 scope.go:117] "RemoveContainer" containerID="5aa697904e6c318b3c3852fa534db294ed31be93bca9791a0e4f7c576eae26c8" Jul 23 19:41:06.429314 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.429296 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:41:06.517523 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517501 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fzlr\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517541 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517593 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517619 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517652 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517681 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.517988 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517749 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.518295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518098 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.518295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.517932 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:41:06.518295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518030 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:41:06.518295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518161 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token\") pod \"f421f22f-49cd-42b5-9949-2eabecde19d3\" (UID: \"f421f22f-49cd-42b5-9949-2eabecde19d3\") " Jul 23 19:41:06.518295 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518036 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:41:06.518614 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518596 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f421f22f-49cd-42b5-9949-2eabecde19d3-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.518684 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518620 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.518684 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518635 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.518790 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518766 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:41:06.518888 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.518869 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:41:06.520245 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.520217 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:41:06.520352 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.520243 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr" (OuterVolumeSpecName: "kube-api-access-2fzlr") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "kube-api-access-2fzlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:41:06.520352 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.520302 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:41:06.520508 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.520483 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f421f22f-49cd-42b5-9949-2eabecde19d3" (UID: "f421f22f-49cd-42b5-9949-2eabecde19d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:41:06.619956 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619926 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.619956 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619950 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.619956 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619959 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.620150 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619968 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f421f22f-49cd-42b5-9949-2eabecde19d3-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.620150 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619976 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:06.620150 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:06.619984 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fzlr\" (UniqueName: \"kubernetes.io/projected/f421f22f-49cd-42b5-9949-2eabecde19d3-kube-api-access-2fzlr\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:41:07.410773 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:07.410696 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" Jul 23 19:41:07.410773 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:07.410725 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8" event={"ID":"f421f22f-49cd-42b5-9949-2eabecde19d3","Type":"ContainerDied","Data":"c3252b19b9bdd015ed445527684a03995df8c1c8bd3479449d1c4d025c8e8426"} Jul 23 19:41:07.411011 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:07.410775 2576 scope.go:117] "RemoveContainer" containerID="77cd1c1ab374ca7e1ec5e1dce91bc924d93e286a93a8487a8c23ce6b0921677d" Jul 23 19:41:07.444282 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:07.444254 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:41:07.448512 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:07.448491 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7e6d6b-openshift-default-6dd5f56df7-jgvj8"] Jul 23 19:41:09.047195 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:09.047165 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" path="/var/lib/kubelet/pods/f421f22f-49cd-42b5-9949-2eabecde19d3/volumes" Jul 23 19:41:09.800297 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:09.800268 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:10.132477 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.132394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:10.467112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.467042 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:41:10.467353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.467342 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:41:10.467405 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.467354 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:41:10.467447 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.467412 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:41:10.467447 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.467419 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:41:10.471797 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.471776 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.474002 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.473987 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4a426a-openshift-default-dockercfg-mpk7s\"" Jul 23 19:41:10.483726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.483690 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:41:10.489070 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.489046 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:10.553514 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553522 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553648 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553638 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553753 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553669 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553753 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553753 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553740 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553767 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.553884 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.553806 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xwpc\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654598 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654564 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xwpc\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654678 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654698 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.654779 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.654791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655172 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.655140 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655290 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.655226 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655345 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.655327 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.655371 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.655641 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.655617 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.656960 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.656940 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.657223 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.657206 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.662621 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.662600 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.662800 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.662783 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xwpc\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc\") pod \"e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.782003 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.781974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:10.913328 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.913285 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:10.934017 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.933995 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:41:10.935941 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:41:10.935917 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74790c50_3ea6_4ce2_b84d_4186322b05b2.slice/crio-315357a76b515a082c7741490e64d36a109e2d1cfe7a909f17c5519cec045d3e WatchSource:0}: Error finding container 315357a76b515a082c7741490e64d36a109e2d1cfe7a909f17c5519cec045d3e: Status 404 returned error can't find the container with id 315357a76b515a082c7741490e64d36a109e2d1cfe7a909f17c5519cec045d3e Jul 23 19:41:10.937892 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.937864 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:10.937986 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.937920 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:10.937986 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:10.937949 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:11.435256 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:11.435219 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerStarted","Data":"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8"} Jul 23 19:41:11.435671 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:11.435306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerStarted","Data":"315357a76b515a082c7741490e64d36a109e2d1cfe7a909f17c5519cec045d3e"} Jul 23 19:41:11.455701 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:11.455656 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podStartSLOduration=1.455641051 podStartE2EDuration="1.455641051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 19:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:41:11.455302219 +0000 UTC m=+2419.033618975" watchObservedRunningTime="2026-07-23 19:41:11.455641051 +0000 UTC m=+2419.033957805" Jul 23 19:41:11.782396 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:11.782361 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:11.787112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:11.787087 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:12.442550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:12.442498 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:12.447403 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:12.447368 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:13.349793 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:13.349724 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:13.438326 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:13.438295 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:13.564567 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:13.564529 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:13.760110 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:13.760054 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:13.999586 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:13.999551 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:14.099002 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:14.098919 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:17.821321 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:17.821276 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:19.498352 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:19.497319 2576 patch_prober.go:28] interesting pod/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:60328->10.132.0.40:15021: read: connection reset by peer" start-of-body= Jul 23 19:41:19.498352 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:19.497395 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:60328->10.132.0.40:15021: read: connection reset by peer" Jul 23 19:41:20.487112 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.487075 2576 generic.go:356] "Generic (PLEG): container finished" podID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerID="9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed" exitCode=0 Jul 23 19:41:20.487301 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.487131 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerDied","Data":"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed"} Jul 23 19:41:20.487540 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.487525 2576 scope.go:117] "RemoveContainer" containerID="9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed" Jul 23 19:41:20.488531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.488500 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:20.488633 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.488584 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:20.488694 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:20.488631 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:21.492898 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:21.492792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerStarted","Data":"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef"} Jul 23 19:41:22.326516 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:22.326462 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:22.326755 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:22.326710 2576 patch_prober.go:28] interesting pod/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 19:41:22.326828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:22.326771 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 19:41:23.326828 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:23.326789 2576 patch_prober.go:28] interesting pod/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 19:41:23.327273 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:23.326868 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 19:41:24.326547 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.326492 2576 patch_prober.go:28] interesting pod/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 19:41:24.326775 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.326574 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 19:41:24.507707 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.507679 2576 generic.go:356] "Generic (PLEG): container finished" podID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerID="d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8" exitCode=0 Jul 23 19:41:24.508208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.507738 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerDied","Data":"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8"} Jul 23 19:41:24.508277 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.508209 2576 scope.go:117] "RemoveContainer" containerID="d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8" Jul 23 19:41:24.509179 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.509153 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:24.509242 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.509218 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:24.509280 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:24.509248 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236236Ki","pods":"250"} Jul 23 19:41:25.330140 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.330111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:25.330377 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.330356 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:25.331154 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.331135 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:41:25.513516 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.513486 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerStarted","Data":"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a"} Jul 23 19:41:25.783168 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.783125 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:25.784332 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.784308 2576 patch_prober.go:28] interesting pod/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 19:41:25.784452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:25.784381 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 19:41:26.782457 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:26.782420 2576 patch_prober.go:28] interesting pod/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 19:41:26.782912 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:26.782498 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 19:41:27.266347 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:27.266313 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:27.366595 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:27.366559 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:27.442502 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:27.442436 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:27.783054 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:27.783017 2576 patch_prober.go:28] interesting pod/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 19:41:27.783526 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:27.783078 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 19:41:28.783259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:28.783223 2576 patch_prober.go:28] interesting pod/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 19:41:28.783695 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:28.783278 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 19:41:29.786436 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:29.786405 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:29.787400 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:29.787380 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:41:34.733129 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:34.733093 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:34.834232 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:34.834198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:36.598511 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:36.598473 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:48.229969 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:48.229931 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:48.327127 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:48.327097 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:52.576412 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:52.576379 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:41:53.187129 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:41:53.187095 2576 scope.go:117] "RemoveContainer" containerID="41593024c1ab337c9b097b06d2b72ce2b625e91daf171157232ea0e518506df7" Jul 23 19:43:00.513812 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:43:00.513776 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:43:03.940239 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:43:03.940198 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:43:04.003667 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:43:04.003633 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:44:09.769721 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:44:09.769689 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:44:10.566208 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:44:10.566179 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:44:14.605054 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:44:14.605024 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:44:36.805817 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:44:36.805779 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:45:00.137392 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.137359 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv"] Jul 23 19:45:00.137895 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.137679 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:45:00.137895 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.137691 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f421f22f-49cd-42b5-9949-2eabecde19d3" containerName="istio-proxy" Jul 23 19:45:00.140523 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.140505 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz"] Jul 23 19:45:00.140635 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.140617 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" Jul 23 19:45:00.142865 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.142847 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ws5pv\"" Jul 23 19:45:00.143391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.143376 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" Jul 23 19:45:00.153469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.153420 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz"] Jul 23 19:45:00.153469 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.153432 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" Jul 23 19:45:00.155735 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.155712 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv"] Jul 23 19:45:00.159677 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.159658 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" Jul 23 19:45:00.301695 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.301667 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv"] Jul 23 19:45:00.305079 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:45:00.305051 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1943e1e5_4919_4e3f_8f0e_b226bfede916.slice/crio-7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3 WatchSource:0}: Error finding container 7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3: Status 404 returned error can't find the container with id 7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3 Jul 23 19:45:00.322736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.322713 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz"] Jul 23 19:45:00.326567 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:45:00.326540 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b8792c_39dc_4882_84a4_0a14091bc4e4.slice/crio-611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8 WatchSource:0}: Error finding container 611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8: Status 404 returned error can't find the container with id 611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8 Jul 23 19:45:00.474289 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.474198 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" event={"ID":"1943e1e5-4919-4e3f-8f0e-b226bfede916","Type":"ContainerStarted","Data":"7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3"} Jul 23 19:45:00.475294 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:00.475272 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" event={"ID":"04b8792c-39dc-4882-84a4-0a14091bc4e4","Type":"ContainerStarted","Data":"611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8"} Jul 23 19:45:01.481499 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:01.481468 2576 generic.go:356] "Generic (PLEG): container finished" podID="1943e1e5-4919-4e3f-8f0e-b226bfede916" containerID="6352ee442131c084ff6adb7b8a42ea565702901270464517ef9758a6b88351bd" exitCode=0 Jul 23 19:45:01.481924 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:01.481551 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" event={"ID":"1943e1e5-4919-4e3f-8f0e-b226bfede916","Type":"ContainerDied","Data":"6352ee442131c084ff6adb7b8a42ea565702901270464517ef9758a6b88351bd"} Jul 23 19:45:01.489190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:01.489159 2576 generic.go:356] "Generic (PLEG): container finished" podID="04b8792c-39dc-4882-84a4-0a14091bc4e4" containerID="4fd8544c10b1c8009ac9817fa2ed71126bd695941963b9ae32cc666901fc444a" exitCode=0 Jul 23 19:45:01.489311 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:01.489213 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" event={"ID":"04b8792c-39dc-4882-84a4-0a14091bc4e4","Type":"ContainerDied","Data":"4fd8544c10b1c8009ac9817fa2ed71126bd695941963b9ae32cc666901fc444a"} Jul 23 19:45:02.641550 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:02.641527 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" Jul 23 19:45:02.644216 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:02.644199 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" Jul 23 19:45:03.500736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.500641 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" event={"ID":"04b8792c-39dc-4882-84a4-0a14091bc4e4","Type":"ContainerDied","Data":"611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8"} Jul 23 19:45:03.500736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.500703 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="611ddcd368677f067a880e11631759de515b1c023d8cd6a31c7e3a99ff253ed8" Jul 23 19:45:03.500736 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.500652 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz" Jul 23 19:45:03.501860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.501816 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" event={"ID":"1943e1e5-4919-4e3f-8f0e-b226bfede916","Type":"ContainerDied","Data":"7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3"} Jul 23 19:45:03.501975 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.501872 2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e7c69ec4dda922e15591460f794e0c2a177bff797c8a54766d73648bb3e56d3" Jul 23 19:45:03.501975 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:03.501826 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv" Jul 23 19:45:17.600525 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:17.600450 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:45:18.375679 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:18.375649 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:45:35.104727 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:35.104692 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:45:53.140209 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:53.140123 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:45:53.143726 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:53.143701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:45:53.227633 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:45:53.227605 2576 scope.go:117] "RemoveContainer" containerID="679eb6b9a7c5b48ac21234f9337812546c1ad1b422f43df7124b8b65ad9fcc76" Jul 23 19:46:03.231702 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:03.231671 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:28.029038 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:28.029003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:28.129341 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:28.129308 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:35.611434 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:35.611398 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:35.712915 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:35.712876 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:49.112858 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:49.112775 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:49.213388 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:49.213356 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:49.558282 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:49.558247 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:46:50.507930 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:46:50.507895 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:03.456486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:03.456449 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:09.849901 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:09.849863 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:09.972432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:09.972399 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:16.028009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:16.027929 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:16.128860 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:16.128805 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:19.627047 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:19.626996 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:19.632711 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:19.632685 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz"] Jul 23 19:47:19.635593 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:19.635572 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d5ee7b-29747265-mdlmz"] Jul 23 19:47:21.047065 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:21.047035 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b8792c-39dc-4882-84a4-0a14091bc4e4" path="/var/lib/kubelet/pods/04b8792c-39dc-4882-84a4-0a14091bc4e4/volumes" Jul 23 19:47:24.251339 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:24.251311 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:24.637571 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:24.637535 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:30.959352 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:30.959317 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:31.014065 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:31.014028 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:31.197126 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:31.197094 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:47:31.197366 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:31.197343 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" containerID="cri-o://5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef" gracePeriod=30 Jul 23 19:47:31.221475 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:31.221394 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:31.437121 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:31.437084 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:32.153204 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:32.153165 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:32.254530 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:32.254488 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:36.452638 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.452613 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:47:36.524160 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524122 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524167 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4v49\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524239 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524263 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524289 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524310 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524362 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524352 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524669 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524388 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524669 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524438 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo\") pod \"8940cb83-eb04-4713-bc28-7d71e680cb26\" (UID: \"8940cb83-eb04-4713-bc28-7d71e680cb26\") " Jul 23 19:47:36.524669 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524448 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:36.524823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524676 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:36.524823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524701 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.524823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524707 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:47:36.524823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.524777 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:36.525170 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.525140 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data" (OuterVolumeSpecName: "istio-data") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:36.526626 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.526595 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:47:36.526626 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.526617 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token" (OuterVolumeSpecName: "istio-token") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:47:36.526764 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.526655 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49" (OuterVolumeSpecName: "kube-api-access-g4v49") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "kube-api-access-g4v49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:47:36.526764 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.526656 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8940cb83-eb04-4713-bc28-7d71e680cb26" (UID: "8940cb83-eb04-4713-bc28-7d71e680cb26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:36.626201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626160 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626196 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626201 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626206 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4v49\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-kube-api-access-g4v49\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626217 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626226 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626235 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8940cb83-eb04-4713-bc28-7d71e680cb26-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626242 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:36.626451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:36.626249 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8940cb83-eb04-4713-bc28-7d71e680cb26-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:37.199645 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.199613 2576 generic.go:356] "Generic (PLEG): container finished" podID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerID="5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef" exitCode=0 Jul 23 19:47:37.199851 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.199666 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" Jul 23 19:47:37.199851 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.199677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerDied","Data":"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef"} Jul 23 19:47:37.199851 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.199723 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg" event={"ID":"8940cb83-eb04-4713-bc28-7d71e680cb26","Type":"ContainerDied","Data":"82d765c6edf27065f2f924fd38c3f77ea4e8cf8a036fd5758661b603c4a8dfcd"} Jul 23 19:47:37.199851 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.199743 2576 scope.go:117] "RemoveContainer" containerID="5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef" Jul 23 19:47:37.209827 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.209808 2576 scope.go:117] "RemoveContainer" containerID="9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed" Jul 23 19:47:37.217527 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.217499 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:47:37.218945 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.218919 2576 scope.go:117] "RemoveContainer" containerID="5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef" Jul 23 19:47:37.219729 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:47:37.219695 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef\": container with ID starting with 5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef not found: ID does not exist" containerID="5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef" Jul 23 19:47:37.219813 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.219733 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef"} err="failed to get container status \"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef\": rpc error: code = NotFound desc = could not find container \"5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef\": container with ID starting with 5b8fa666749fbeee4926b5bab01634cacb419d6ca1162e9831df6f14816ad6ef not found: ID does not exist" Jul 23 19:47:37.219813 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.219759 2576 scope.go:117] "RemoveContainer" containerID="9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed" Jul 23 19:47:37.220209 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:47:37.220186 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed\": container with ID starting with 9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed not found: ID does not exist" containerID="9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed" Jul 23 19:47:37.220285 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.220212 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed"} err="failed to get container status \"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed\": rpc error: code = NotFound desc = could not find container \"9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed\": container with ID starting with 9560d12bd5ad28ecda44ab24ebd9ab0de96c4a26e86100b16b0b4efe3f312fed not found: ID does not exist" Jul 23 19:47:37.221439 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:37.221421 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d5ee7b-openshift-default-89d579ddc-dzrjg"] Jul 23 19:47:39.047618 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:39.047585 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" path="/var/lib/kubelet/pods/8940cb83-eb04-4713-bc28-7d71e680cb26/volumes" Jul 23 19:47:41.947452 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:41.947388 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:41.951761 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:41.951736 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv"] Jul 23 19:47:41.954432 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:41.954410 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4a426a-29747265-mbkxv"] Jul 23 19:47:43.046982 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:43.046953 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1943e1e5-4919-4e3f-8f0e-b226bfede916" path="/var/lib/kubelet/pods/1943e1e5-4919-4e3f-8f0e-b226bfede916/volumes" Jul 23 19:47:46.629937 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:46.629820 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:47.010096 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:47.010064 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:53.167451 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.167417 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:53.225467 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.225423 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:53.407210 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.407176 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:47:53.407419 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.407381 2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" containerID="cri-o://961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a" gracePeriod=30 Jul 23 19:47:53.420098 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.420003 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:53.687040 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:53.686950 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:47:58.663974 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.663949 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:47:58.819319 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819284 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819340 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819358 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xwpc\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819391 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819413 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819436 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819489 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819469 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.819920 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819889 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 19:47:58.820009 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.819968 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:58.820086 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820055 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.820149 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820131 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket\") pod \"74790c50-3ea6-4ce2-b84d-4186322b05b2\" (UID: \"74790c50-3ea6-4ce2-b84d-4186322b05b2\") " Jul 23 19:47:58.820531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820218 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:58.820644 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820543 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:58.821885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820910 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-credential-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.821885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820931 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74790c50-3ea6-4ce2-b84d-4186322b05b2-istiod-ca-cert\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.821885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820949 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-socket\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.821885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.820966 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-workload-certs\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.821885 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.821170 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:58.822394 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.822358 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 19:47:58.827558 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.822594 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:47:58.827558 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.822731 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 19:47:58.827558 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.823936 2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc" (OuterVolumeSpecName: "kube-api-access-6xwpc") pod "74790c50-3ea6-4ce2-b84d-4186322b05b2" (UID: "74790c50-3ea6-4ce2-b84d-4186322b05b2"). InnerVolumeSpecName "kube-api-access-6xwpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 19:47:58.922270 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.922226 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-token\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.922270 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.922266 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xwpc\" (UniqueName: \"kubernetes.io/projected/74790c50-3ea6-4ce2-b84d-4186322b05b2-kube-api-access-6xwpc\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.922270 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.922280 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-podinfo\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.922528 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.922291 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-data\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:58.922528 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:58.922303 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74790c50-3ea6-4ce2-b84d-4186322b05b2-istio-envoy\") on node \"ip-10-0-139-123.ec2.internal\" DevicePath \"\"" Jul 23 19:47:59.302047 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.302011 2576 generic.go:356] "Generic (PLEG): container finished" podID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerID="961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a" exitCode=0 Jul 23 19:47:59.302217 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.302072 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" Jul 23 19:47:59.302217 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.302098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerDied","Data":"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a"} Jul 23 19:47:59.302217 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.302146 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5" event={"ID":"74790c50-3ea6-4ce2-b84d-4186322b05b2","Type":"ContainerDied","Data":"315357a76b515a082c7741490e64d36a109e2d1cfe7a909f17c5519cec045d3e"} Jul 23 19:47:59.302217 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.302167 2576 scope.go:117] "RemoveContainer" containerID="961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a" Jul 23 19:47:59.310721 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.310700 2576 scope.go:117] "RemoveContainer" containerID="d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8" Jul 23 19:47:59.318445 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.318423 2576 scope.go:117] "RemoveContainer" containerID="961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a" Jul 23 19:47:59.318752 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:47:59.318724 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a\": container with ID starting with 961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a not found: ID does not exist" containerID="961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a" Jul 23 19:47:59.318896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.318760 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a"} err="failed to get container status \"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a\": rpc error: code = NotFound desc = could not find container \"961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a\": container with ID starting with 961cfe0632f5e88ae036f15d6d2176c02ea2634e0e72a0931d6c546243f54c0a not found: ID does not exist" Jul 23 19:47:59.318896 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.318779 2576 scope.go:117] "RemoveContainer" containerID="d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8" Jul 23 19:47:59.319474 ip-10-0-139-123 kubenswrapper[2576]: E0723 19:47:59.319353 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8\": container with ID starting with d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8 not found: ID does not exist" containerID="d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8" Jul 23 19:47:59.319474 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.319392 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8"} err="failed to get container status \"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8\": rpc error: code = NotFound desc = could not find container \"d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8\": container with ID starting with d092a7f1e8d388fc13e13e248a4193d52971d037413ccb77d5d59ec22871e9f8 not found: ID does not exist" Jul 23 19:47:59.320745 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.320725 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:47:59.324973 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.324953 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4a426a-openshift-default-58556bdcc9-6qnn5"] Jul 23 19:47:59.889946 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.889918 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-cw9gw_5ab76fc3-92e8-4941-a838-fed7e74da552/copy-manifests/0.log" Jul 23 19:47:59.910156 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:47:59.910131 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-cw9gw_5ab76fc3-92e8-4941-a838-fed7e74da552/manager/0.log" Jul 23 19:48:00.486427 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:00.486390 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747235-wlrrc_c9b212f0-9dc9-484c-8c56-ef1739ebd2d6/cleanup/0.log" Jul 23 19:48:00.596728 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:00.596690 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747250-55ztd_98865747-66df-4dcf-b1bf-a7d6d0b3abdb/cleanup/0.log" Jul 23 19:48:01.046908 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:01.046870 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" path="/var/lib/kubelet/pods/74790c50-3ea6-4ce2-b84d-4186322b05b2/volumes" Jul 23 19:48:03.148604 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:03.148572 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rksmv_014d8ce8-f7f0-4b96-990c-e69e311ded73/istio-proxy/0.log" Jul 23 19:48:03.257771 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:03.257741 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-lt8fn_2217e5d1-03bb-444a-a812-fac50591dcee/discovery/0.log" Jul 23 19:48:03.590644 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:03.588289 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/1.log" Jul 23 19:48:03.929108 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:03.929026 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c889d5d-pjpft_5bfefa19-7c7e-4f52-a53e-a473d73137a5/router/0.log" Jul 23 19:48:04.261101 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:04.261075 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xx6x4_f819db0f-816d-47db-b3ef-65ea0e3554b1/main/0.log" Jul 23 19:48:10.241156 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:10.241115 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:48:11.630126 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:11.630092 2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" limit=1024 Jul 23 19:48:13.246743 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:13.246716 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2q6nq_8c29a651-9e48-4ff9-842c-ccd602552695/global-pull-secret-syncer/0.log" Jul 23 19:48:13.373486 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:13.373454 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lwp75_a0c70287-dd25-4cd7-b656-93a14d7605ef/konnectivity-agent/0.log" Jul 23 19:48:13.501411 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:13.501328 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-123.ec2.internal_a63dd2beb5b4bbbe932b9b8b6353efdf/haproxy/0.log" Jul 23 19:48:20.802637 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:20.802604 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-s7krv_e4dd8699-497d-4f9e-988f-f688f3aafd46/cluster-monitoring-operator/0.log" Jul 23 19:48:20.976187 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:20.976157 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6glc9_1b9ef68b-711e-4bcd-b405-99db0b1ce37e/node-exporter/0.log" Jul 23 19:48:20.997583 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:20.997558 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6glc9_1b9ef68b-711e-4bcd-b405-99db0b1ce37e/kube-rbac-proxy/0.log" Jul 23 19:48:21.019535 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:21.019511 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6glc9_1b9ef68b-711e-4bcd-b405-99db0b1ce37e/init-textfile/0.log" Jul 23 19:48:23.370823 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.370788 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg"] Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371251 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371272 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371301 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371309 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371319 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b8792c-39dc-4882-84a4-0a14091bc4e4" containerName="cleanup" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371328 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b8792c-39dc-4882-84a4-0a14091bc4e4" containerName="cleanup" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371337 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1943e1e5-4919-4e3f-8f0e-b226bfede916" containerName="cleanup" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371344 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1943e1e5-4919-4e3f-8f0e-b226bfede916" containerName="cleanup" Jul 23 19:48:23.371353 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371355 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371364 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371441 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8940cb83-eb04-4713-bc28-7d71e680cb26" containerName="istio-proxy" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371455 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1943e1e5-4919-4e3f-8f0e-b226bfede916" containerName="cleanup" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371465 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371475 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b8792c-39dc-4882-84a4-0a14091bc4e4" containerName="cleanup" Jul 23 19:48:23.371792 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.371482 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="74790c50-3ea6-4ce2-b84d-4186322b05b2" containerName="istio-proxy" Jul 23 19:48:23.374755 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.374733 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.376972 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.376919 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ctxw\"/\"openshift-service-ca.crt\"" Jul 23 19:48:23.377903 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.377883 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ctxw\"/\"default-dockercfg-vxjpx\"" Jul 23 19:48:23.377993 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.377936 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ctxw\"/\"kube-root-ca.crt\"" Jul 23 19:48:23.382873 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.382850 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg"] Jul 23 19:48:23.414772 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.414742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-lib-modules\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.414950 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.414782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-podres\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.414950 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.414823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-sys\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.414950 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.414866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-proc\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.414950 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.414890 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb6kg\" (UniqueName: \"kubernetes.io/projected/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-kube-api-access-cb6kg\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-sys\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515678 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515680 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-proc\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb6kg\" (UniqueName: \"kubernetes.io/projected/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-kube-api-access-cb6kg\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-lib-modules\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515781 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-proc\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-podres\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515781 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-sys\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515931 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-podres\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.515949 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.515932 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-lib-modules\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.524380 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.524357 2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb6kg\" (UniqueName: \"kubernetes.io/projected/a112c67f-ecf8-44ac-9de1-a4c5f6b0e899-kube-api-access-cb6kg\") pod \"perf-node-gather-daemonset-nzxvg\" (UID: \"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899\") " pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.584952 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.584917 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/1.log" Jul 23 19:48:23.589563 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.589541 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-29gmr_57d361da-ffa5-491c-8fa8-725e8a97513b/console-operator/2.log" Jul 23 19:48:23.685377 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.685295 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:23.814751 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:23.814731 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg"] Jul 23 19:48:23.817035 ip-10-0-139-123 kubenswrapper[2576]: W0723 19:48:23.817006 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda112c67f_ecf8_44ac_9de1_a4c5f6b0e899.slice/crio-e64690cde5318cd2f1bf3d3b8512837a04ea3674d57d1d0f4e2c712d61324f77 WatchSource:0}: Error finding container e64690cde5318cd2f1bf3d3b8512837a04ea3674d57d1d0f4e2c712d61324f77: Status 404 returned error can't find the container with id e64690cde5318cd2f1bf3d3b8512837a04ea3674d57d1d0f4e2c712d61324f77 Jul 23 19:48:24.417979 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:24.417942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" event={"ID":"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899","Type":"ContainerStarted","Data":"8e1c8634e4e764d3458fdb93419c8af5f5992801ce876ad07b1218e5e48f181f"} Jul 23 19:48:24.418476 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:24.417993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" event={"ID":"a112c67f-ecf8-44ac-9de1-a4c5f6b0e899","Type":"ContainerStarted","Data":"e64690cde5318cd2f1bf3d3b8512837a04ea3674d57d1d0f4e2c712d61324f77"} Jul 23 19:48:24.418476 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:24.418038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:24.435321 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:24.435243 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" podStartSLOduration=1.435075156 podStartE2EDuration="1.435075156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 19:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 19:48:24.434089886 +0000 UTC m=+2852.012406640" watchObservedRunningTime="2026-07-23 19:48:24.435075156 +0000 UTC m=+2852.013391913" Jul 23 19:48:24.765320 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:24.765293 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qbdwv_e6a1a6d3-7ed0-48ad-a745-fa49b2aef942/volume-data-source-validator/0.log" Jul 23 19:48:25.808329 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:25.808296 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rsv86_69e66d5f-e0db-426b-a333-a7f9243f1745/dns/0.log" Jul 23 19:48:25.829119 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:25.829091 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rsv86_69e66d5f-e0db-426b-a333-a7f9243f1745/kube-rbac-proxy/0.log" Jul 23 19:48:25.905755 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:25.905728 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ct8h8_a956c0a0-2420-4040-913e-d714b4dc5418/dns-node-resolver/0.log" Jul 23 19:48:26.494221 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:26.494195 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4xq9v_f9582889-69da-4fc1-9340-3998fd371a39/node-ca/0.log" Jul 23 19:48:27.508485 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:27.508455 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rksmv_014d8ce8-f7f0-4b96-990c-e69e311ded73/istio-proxy/0.log" Jul 23 19:48:27.631435 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:27.631406 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-lt8fn_2217e5d1-03bb-444a-a812-fac50591dcee/discovery/0.log" Jul 23 19:48:27.703427 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:27.703389 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/0.log" Jul 23 19:48:27.729661 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:27.729635 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/1.log" Jul 23 19:48:27.813190 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:27.813113 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c889d5d-pjpft_5bfefa19-7c7e-4f52-a53e-a473d73137a5/router/0.log" Jul 23 19:48:28.491342 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:28.491317 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-54kmk_8e461165-4a68-4b19-9e61-10616c8eed5d/serve-healthcheck-canary/0.log" Jul 23 19:48:28.997397 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:28.997365 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-wmcln_cee55273-2d39-4ff5-81dc-534545313734/insights-operator/0.log" Jul 23 19:48:28.997816 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:28.997794 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-wmcln_cee55273-2d39-4ff5-81dc-534545313734/insights-operator/1.log" Jul 23 19:48:29.017671 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:29.017652 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c8jcn_2ba43c2c-a266-490e-8537-fd1d88a0b7d9/kube-rbac-proxy/0.log" Jul 23 19:48:29.040561 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:29.040534 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c8jcn_2ba43c2c-a266-490e-8537-fd1d88a0b7d9/exporter/0.log" Jul 23 19:48:29.061391 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:29.061365 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-c8jcn_2ba43c2c-a266-490e-8537-fd1d88a0b7d9/extractor/0.log" Jul 23 19:48:30.432531 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:30.432503 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ctxw/perf-node-gather-daemonset-nzxvg" Jul 23 19:48:31.860049 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:31.860018 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-cw9gw_5ab76fc3-92e8-4941-a838-fed7e74da552/manager/0.log" Jul 23 19:48:31.878970 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:31.878947 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6657c8cb8d-cw9gw_5ab76fc3-92e8-4941-a838-fed7e74da552/copy-manifests/0.log" Jul 23 19:48:33.341651 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.341563 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rksmv_014d8ce8-f7f0-4b96-990c-e69e311ded73/istio-proxy/0.log" Jul 23 19:48:33.389680 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.389652 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-779d54d68-kghst_39b15187-ad67-41b9-88e4-828d8e98101d/manager/0.log" Jul 23 19:48:33.465639 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.465607 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-lt8fn_2217e5d1-03bb-444a-a812-fac50591dcee/discovery/0.log" Jul 23 19:48:33.511744 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.511706 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/0.log" Jul 23 19:48:33.540308 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.540280 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/1.log" Jul 23 19:48:33.595371 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:33.595286 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c889d5d-pjpft_5bfefa19-7c7e-4f52-a53e-a473d73137a5/router/0.log" Jul 23 19:48:34.777738 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:34.777712 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678rksmv_014d8ce8-f7f0-4b96-990c-e69e311ded73/istio-proxy/0.log" Jul 23 19:48:34.896242 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:34.896161 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-lt8fn_2217e5d1-03bb-444a-a812-fac50591dcee/discovery/0.log" Jul 23 19:48:34.947781 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:34.947751 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/0.log" Jul 23 19:48:34.972603 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:34.972575 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-2s6ms_3276c746-37b7-40ff-b653-7ede69c57ae6/istio-proxy/1.log" Jul 23 19:48:35.034259 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:35.034231 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9c889d5d-pjpft_5bfefa19-7c7e-4f52-a53e-a473d73137a5/router/0.log" Jul 23 19:48:38.781866 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:38.781811 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-ks8td_8c8dddf0-949c-4228-a169-88a50e857fb2/kube-storage-version-migrator-operator/1.log" Jul 23 19:48:38.782737 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:38.782720 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-ks8td_8c8dddf0-949c-4228-a169-88a50e857fb2/kube-storage-version-migrator-operator/0.log" Jul 23 19:48:39.783138 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:39.783108 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6s2m7_5bb7b0bd-5164-4341-8948-08dd57301b5b/kube-multus/0.log" Jul 23 19:48:40.147461 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.147382 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/kube-multus-additional-cni-plugins/0.log" Jul 23 19:48:40.170214 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.170187 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/egress-router-binary-copy/0.log" Jul 23 19:48:40.191752 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.191724 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/cni-plugins/0.log" Jul 23 19:48:40.213611 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.213587 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/bond-cni-plugin/0.log" Jul 23 19:48:40.235386 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.235365 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/routeoverride-cni/0.log" Jul 23 19:48:40.257074 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.257047 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/whereabouts-cni-bincopy/0.log" Jul 23 19:48:40.279146 ip-10-0-139-123 kubenswrapper[2576]: I0723 19:48:40.279119 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44x8_892756f1-263f-46af-b14f-a38baa295c80/whereabouts-cni/0.log"