Jul 10 20:34:50.988154 ip-10-0-137-185 systemd[1]: Starting Kubernetes Kubelet... Jul 10 20:34:51.362878 ip-10-0-137-185 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:34:51.362878 ip-10-0-137-185 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 20:34:51.362878 ip-10-0-137-185 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:34:51.362878 ip-10-0-137-185 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 20:34:51.362878 ip-10-0-137-185 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 20:34:51.363770 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.363643 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 20:34:51.371352 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371325 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.371352 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371343 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.371352 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371349 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.371352 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371354 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.371352 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371358 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371361 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371365 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371367 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371370 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371373 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371375 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371378 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371381 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371384 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371386 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371389 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371392 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371395 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371398 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371400 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371403 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371405 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371408 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371410 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.371545 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371413 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371416 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371419 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371421 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371424 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371426 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371429 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371432 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371437 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371441 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371444 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371447 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371450 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371453 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371456 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371459 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371462 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371464 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371467 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371470 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.372033 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371473 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371475 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371478 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371481 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371483 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371487 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371489 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371492 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371495 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371497 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371500 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371504 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371507 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371510 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371513 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371515 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371518 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371521 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371523 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371526 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.372527 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371528 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371530 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371533 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371535 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371539 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371541 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371546 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371549 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371552 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371555 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371557 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371560 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371563 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371565 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371569 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371571 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371574 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371576 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371579 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371582 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.373028 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371584 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371587 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371589 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371592 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371594 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371597 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371599 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371602 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371604 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371607 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371609 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371612 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371615 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371617 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.371619 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372366 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372374 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372377 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372382 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372385 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.373656 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372388 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372391 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372394 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372396 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372399 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372402 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372404 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372408 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372410 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372413 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372416 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372420 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372423 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372425 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372428 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372430 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372433 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372436 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372438 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372441 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.374184 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372443 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372446 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372449 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372451 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372454 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372456 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372459 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372461 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372466 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372469 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372474 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372477 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372481 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372484 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372487 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372489 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372492 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372495 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.374666 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372497 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372500 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372503 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372506 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372511 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372514 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372517 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372520 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372523 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372526 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372528 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372531 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372534 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372537 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372540 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372542 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372544 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372547 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372550 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.375149 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372553 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372556 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372558 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372561 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372563 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372567 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372570 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372572 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372575 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372577 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372580 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372583 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372585 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372588 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372591 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372593 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372596 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372598 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372601 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372603 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.375598 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372606 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372609 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372612 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372615 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372618 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372620 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372623 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372625 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372628 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372630 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372633 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372635 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372638 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372640 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372643 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372645 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.372647 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373034 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373040 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373044 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.376105 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373047 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373051 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373053 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373057 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373060 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373062 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373066 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373069 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373073 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373076 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373079 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373081 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373084 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373087 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373090 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373093 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373096 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373099 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373102 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.376577 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373105 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373107 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373110 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373112 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373114 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373117 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373120 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373122 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373125 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373127 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373130 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373133 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373135 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373138 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373140 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373143 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373146 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373148 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373150 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373153 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.377030 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373156 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373158 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373161 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373163 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373166 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373168 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373171 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373174 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373177 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373180 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373183 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373186 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373188 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373191 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373193 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373196 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373198 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373201 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373204 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373207 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.377513 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373209 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373211 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373215 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373217 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373221 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373225 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373228 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373231 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373233 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373236 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373238 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373241 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373243 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373246 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373249 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373251 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373254 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373258 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373261 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.378007 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373263 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373267 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373269 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373272 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373274 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373277 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373279 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373282 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373284 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373287 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373289 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373292 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373295 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373298 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373300 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373303 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373305 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.373308 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373383 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373391 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373396 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 20:34:51.378472 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373401 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373406 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373409 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373413 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373417 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373420 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373423 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373427 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373431 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373435 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373438 2566 flags.go:64] FLAG: --cgroup-root="" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373441 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373444 2566 flags.go:64] FLAG: --client-ca-file="" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373448 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373451 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373455 2566 flags.go:64] FLAG: --cluster-domain="" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373458 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373461 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373464 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373468 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373471 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373474 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373480 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 20:34:51.378988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373483 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373486 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373489 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373492 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373495 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373498 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373502 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373505 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373508 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373511 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373514 2566 flags.go:64] FLAG: --enable-server="true" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373517 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373521 2566 flags.go:64] FLAG: --event-burst="100" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373524 2566 flags.go:64] FLAG: --event-qps="50" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373527 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373530 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373533 2566 flags.go:64] FLAG: --eviction-hard="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373537 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373541 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373544 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373547 2566 flags.go:64] FLAG: --eviction-soft="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373549 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373552 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373556 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373559 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 20:34:51.379522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373562 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373565 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373568 2566 flags.go:64] FLAG: --feature-gates="" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373571 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373575 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373578 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373582 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373585 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373589 2566 flags.go:64] FLAG: --help="false" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373592 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373595 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373598 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373601 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373604 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373608 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373610 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373613 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373616 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373619 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373622 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373625 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373628 2566 flags.go:64] FLAG: --kube-reserved="" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373631 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373634 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 20:34:51.380199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373637 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373641 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373644 2566 flags.go:64] FLAG: --lock-file="" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373647 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373649 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373652 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373658 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373661 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373665 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373668 2566 flags.go:64] FLAG: --logging-format="text" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373671 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373674 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373677 2566 flags.go:64] FLAG: --manifest-url="" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373680 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373685 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373688 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373692 2566 flags.go:64] FLAG: --max-pods="110" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373695 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373697 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373700 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373703 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373706 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373709 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373712 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373719 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 20:34:51.380788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373722 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373725 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373728 2566 flags.go:64] FLAG: --pod-cidr="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373731 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373736 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373739 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373742 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373745 2566 flags.go:64] FLAG: --port="10250" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373749 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373752 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09b6c082a00dde585" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373756 2566 flags.go:64] FLAG: --qos-reserved="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373758 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373761 2566 flags.go:64] FLAG: --register-node="true" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373764 2566 flags.go:64] FLAG: --register-with-taints="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373768 2566 flags.go:64] FLAG: --registry-burst="10" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373778 2566 flags.go:64] FLAG: --registry-qps="5" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373781 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373783 2566 flags.go:64] FLAG: --reserved-memory="" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373787 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373790 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373793 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373796 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373799 2566 flags.go:64] FLAG: --runonce="false" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373802 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373820 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 20:34:51.381399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373824 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373827 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373830 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373833 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373836 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373839 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373842 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373845 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373848 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373850 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373853 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373856 2566 flags.go:64] FLAG: --system-cgroups="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373859 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373865 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373867 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373872 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373876 2566 flags.go:64] FLAG: --tls-min-version="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373879 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373882 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373885 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373888 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373892 2566 flags.go:64] FLAG: --v="2" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373896 2566 flags.go:64] FLAG: --version="false" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373901 2566 flags.go:64] FLAG: --vmodule="" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373905 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 20:34:51.382016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.373908 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374025 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374029 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374032 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374035 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374038 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374040 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374043 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374046 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374048 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374051 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374054 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374056 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374059 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374061 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374064 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374066 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374069 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374071 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374074 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.382613 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374076 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374079 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374083 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374087 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374090 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374093 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374096 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374099 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374101 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374104 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374106 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374109 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374112 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374114 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374116 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374119 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374121 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374124 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374127 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.383094 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374129 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374132 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374135 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374137 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374140 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374142 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374145 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374148 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374150 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374154 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374158 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374162 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374165 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374168 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374171 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374176 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374179 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374181 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374184 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.383550 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374187 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374190 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374193 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374195 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374198 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374201 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374204 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374206 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374209 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374211 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374216 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374219 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374222 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374224 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374227 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374229 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374232 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374234 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374237 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374239 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.384066 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374242 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374244 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374247 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374249 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374252 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374254 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374257 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374260 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374264 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374266 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374269 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374272 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374274 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374277 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374279 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374282 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374284 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374287 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374289 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374292 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.384540 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374295 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.374297 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.374918 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382210 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382226 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382247 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382252 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382309 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382314 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382318 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382321 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382324 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382326 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382329 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382332 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.385016 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382334 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382337 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382340 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382343 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382345 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382348 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382351 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382353 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382356 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382358 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382361 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382364 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382366 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382369 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382372 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382374 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382377 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382380 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382383 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382385 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.385389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382388 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382392 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382395 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382398 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382400 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382403 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382405 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382408 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382412 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382416 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382419 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382422 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382425 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382428 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382431 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382434 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382436 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382439 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382442 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382444 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.385872 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382447 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382449 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382452 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382455 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382457 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382460 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382462 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382465 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382467 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382471 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382473 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382476 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382479 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382482 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382485 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382488 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382490 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382493 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382496 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382498 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.386355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382501 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382505 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382508 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382511 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382514 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382516 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382519 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382523 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382526 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382529 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382532 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382535 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382538 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382540 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382543 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382546 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382549 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382552 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382554 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.386846 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382557 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382559 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382562 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382565 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382568 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382571 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382574 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382576 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382579 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382581 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382584 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382587 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382592 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382705 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382710 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 20:34:51.387283 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382713 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382716 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382719 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382723 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382726 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382730 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382732 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382735 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382737 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382740 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382742 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382745 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382747 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382750 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382752 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382755 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382758 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382760 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382763 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382765 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 20:34:51.387642 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382768 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382771 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382773 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382776 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382779 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382781 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382784 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382787 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382789 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382792 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382794 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382797 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382799 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382802 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382820 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382825 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382829 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382832 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382835 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382838 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 20:34:51.388147 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382840 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382844 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382848 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382850 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382853 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382856 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382859 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382861 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382864 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382867 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382869 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382872 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382874 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382877 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382880 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382884 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382887 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382889 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382892 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 20:34:51.388618 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382894 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382897 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382900 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382902 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382905 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382907 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382910 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382912 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382915 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382917 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382920 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382923 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382925 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382927 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382930 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382932 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382935 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382937 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382940 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 20:34:51.389227 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382943 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382947 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382950 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382952 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382955 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382957 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382960 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382962 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382965 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382968 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382971 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382974 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382976 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382979 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382981 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382984 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382986 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382989 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 20:34:51.389679 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:51.382991 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.382996 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.383098 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.385442 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.387246 2566 server.go:1019] "Starting client certificate rotation" Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.387346 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.387377 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 20:34:51.390117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.389631 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:34:51.391242 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.391225 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 20:34:51.399333 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.399315 2566 log.go:25] "Validated CRI v1 runtime API" Jul 10 20:34:51.402793 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.402773 2566 log.go:25] "Validated CRI v1 image API" Jul 10 20:34:51.403793 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.403775 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 20:34:51.405893 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.405874 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7da875d7-3fbd-4ab7-a37f-b48025aef656:/dev/nvme0n1p4 b63391b7-3e52-4856-a828-80906ef3fe31:/dev/nvme0n1p3] Jul 10 20:34:51.405957 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.405892 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 20:34:51.411584 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.411463 2566 manager.go:216] Machine: {Timestamp:2026-07-10 20:34:51.410230082 +0000 UTC m=+0.294061416 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099355 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec222d876494c94bf3abcd86a022cbf7 SystemUUID:ec222d87-6494-c94b-f3ab-cd86a022cbf7 BootID:ff5ab808-dd98-4cff-97d8-442388f87c85 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:98:cf:a2:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:98:cf:a2:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:e0:da:7d:85:ad Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 20:34:51.411584 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.411568 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 20:34:51.411744 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.411683 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 20:34:51.413419 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.413405 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:34:51.414514 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.414492 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 20:34:51.414667 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.414519 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 20:34:51.415255 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.415244 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 20:34:51.415311 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.415258 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 20:34:51.415311 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.415270 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:34:51.415933 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.415922 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 20:34:51.417623 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.417614 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 20:34:51.420856 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.420845 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:34:51.420991 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.420982 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 20:34:51.421021 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421011 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 10 20:34:51.421060 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421020 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 20:34:51.421060 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421035 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 20:34:51.421060 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421045 2566 kubelet.go:398] "Adding apiserver pod source" Jul 10 20:34:51.421060 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421053 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 20:34:51.421406 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421395 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:34:51.421447 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.421415 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 20:34:51.423410 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.423391 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 20:34:51.425862 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.425848 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 20:34:51.425921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.425878 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 20:34:51.426638 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426627 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426642 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426648 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426654 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426663 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426669 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426676 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 20:34:51.426679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426681 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 20:34:51.426873 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426688 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 20:34:51.426873 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426697 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 20:34:51.426873 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426718 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 20:34:51.426873 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.426727 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 20:34:51.427421 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.427411 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 20:34:51.427450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.427421 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 20:34:51.430327 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.430304 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 20:34:51.430908 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.430896 2566 server.go:1268] "Started kubelet" Jul 10 20:34:51.430960 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.430924 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 20:34:51.430960 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.430922 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 20:34:51.431048 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.430976 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 20:34:51.431081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.431057 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 20:34:51.432959 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.432937 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 20:34:51.433879 ip-10-0-137-185 systemd[1]: Started Kubernetes Kubelet. Jul 10 20:34:51.434541 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.434383 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 20:34:51.435717 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.435695 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 10 20:34:51.436504 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.436483 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptjf5" Jul 10 20:34:51.438000 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.437983 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 20:34:51.439188 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.438516 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-185.ec2.internal.18c107f88806dea9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-185.ec2.internal,UID:ip-10-0-137-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-185.ec2.internal,},FirstTimestamp:2026-07-10 20:34:51.430862505 +0000 UTC m=+0.314693840,LastTimestamp:2026-07-10 20:34:51.430862505 +0000 UTC m=+0.314693840,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-185.ec2.internal,}" Jul 10 20:34:51.442184 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442164 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 20:34:51.442184 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442176 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 20:34:51.442323 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.442303 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.442646 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442629 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 20:34:51.442646 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442648 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 20:34:51.442777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442628 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 20:34:51.442777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442726 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 10 20:34:51.442777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.442735 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 10 20:34:51.443146 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443127 2566 factory.go:153] Registering CRI-O factory Jul 10 20:34:51.443146 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443147 2566 factory.go:223] Registration of the crio container factory successfully Jul 10 20:34:51.443285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443201 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 20:34:51.443285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443212 2566 factory.go:55] Registering systemd factory Jul 10 20:34:51.443285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443221 2566 factory.go:223] Registration of the systemd container factory successfully Jul 10 20:34:51.443285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443243 2566 factory.go:103] Registering Raw factory Jul 10 20:34:51.443285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443258 2566 manager.go:1196] Started watching for new ooms in manager Jul 10 20:34:51.443560 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.443546 2566 manager.go:349] Starting recovery of all containers Jul 10 20:34:51.444799 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.444780 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptjf5" Jul 10 20:34:51.451459 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.451435 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:51.453081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.453066 2566 manager.go:354] Recovery completed Jul 10 20:34:51.454597 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.454557 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.457653 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.457641 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.459859 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.459845 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.459928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.459877 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.459928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.459889 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.460358 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.460345 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 20:34:51.460400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.460359 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.460400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.460359 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 20:34:51.460470 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.460407 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 10 20:34:51.461411 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461398 2566 policy_none.go:49] "None policy: Start" Jul 10 20:34:51.461484 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461416 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 20:34:51.461484 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461430 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 20:34:51.461484 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461442 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.461484 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461471 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.461484 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.461484 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.462561 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.462550 2566 policy_none.go:47] "Start" Jul 10 20:34:51.500492 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.500329 2566 manager.go:337] "Starting Device Plugin manager" Jul 10 20:34:51.500579 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.500546 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 20:34:51.500579 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.500561 2566 server.go:87] "Starting device plugin registration server" Jul 10 20:34:51.500840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.500827 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 20:34:51.500874 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.500845 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 20:34:51.500945 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.500929 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 20:34:51.501024 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.501009 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 20:34:51.501024 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.501021 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 20:34:51.501463 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.501446 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 20:34:51.501527 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.501486 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.588956 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.588924 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 20:34:51.590148 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.590133 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 20:34:51.590220 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.590168 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 20:34:51.590220 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.590195 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 20:34:51.590301 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.590242 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 20:34:51.592384 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.592363 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:51.601860 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.601838 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.603554 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.603537 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.603644 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.603567 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.603644 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.603577 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.603644 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.603601 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.609719 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.609705 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.609780 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.609729 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-185.ec2.internal\": node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.627086 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.627037 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.691400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.691359 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal"] Jul 10 20:34:51.691535 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.691442 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.692424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.692408 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.692488 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.692438 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.692488 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.692452 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.694055 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694042 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.694192 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694176 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.694242 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694228 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.694801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694781 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.694801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694803 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.694801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694813 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.694988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694827 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.694988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694844 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.694988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.694856 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.696112 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.696099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.696160 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.696124 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 20:34:51.696771 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.696754 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientMemory" Jul 10 20:34:51.696862 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.696784 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 20:34:51.696862 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.696796 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeHasSufficientPID" Jul 10 20:34:51.726605 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.726585 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.727341 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.727327 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.731460 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.731444 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-185.ec2.internal\" not found" node="ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.828114 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.828085 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.844153 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.844104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.844153 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.844152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.844153 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.844172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.929074 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:51.928990 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:51.944449 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.944522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944455 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.944522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944471 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.944522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.944617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29da48fefeff7a1251df734706134371-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal\" (UID: \"29da48fefeff7a1251df734706134371\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:51.944617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:51.944539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b338db47c4937c088ae739177bb426e7-config\") pod \"kube-apiserver-proxy-ip-10-0-137-185.ec2.internal\" (UID: \"b338db47c4937c088ae739177bb426e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:52.033227 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.033188 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.033321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.033243 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:52.035598 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.035581 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:52.135970 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.135923 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.236708 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.236621 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.343902 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.343866 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.389611 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.389584 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 20:34:52.390110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.389767 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 20:34:52.390110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.389770 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 20:34:52.444722 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.444696 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.448540 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.448514 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 20:29:52 +0000 UTC" deadline="2028-04-03 20:37:49.040144186 +0000 UTC" Jul 10 20:34:52.448591 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.448540 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15192h2m56.591607015s" Jul 10 20:34:52.545881 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.545239 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.598702 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:52.598667 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb338db47c4937c088ae739177bb426e7.slice/crio-b1cea374efda3c4f4552624f3eb67c54a6f9a217fe65d0bdec0740b0e4ea8d51 WatchSource:0}: Error finding container b1cea374efda3c4f4552624f3eb67c54a6f9a217fe65d0bdec0740b0e4ea8d51: Status 404 returned error can't find the container with id b1cea374efda3c4f4552624f3eb67c54a6f9a217fe65d0bdec0740b0e4ea8d51 Jul 10 20:34:52.599784 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:52.599755 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29da48fefeff7a1251df734706134371.slice/crio-abffd69d8c93948a35c79748737f1d17d57d728bf5846cdd529851e51578787e WatchSource:0}: Error finding container abffd69d8c93948a35c79748737f1d17d57d728bf5846cdd529851e51578787e: Status 404 returned error can't find the container with id abffd69d8c93948a35c79748737f1d17d57d728bf5846cdd529851e51578787e Jul 10 20:34:52.603730 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.603710 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:34:52.645902 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.645869 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.747504 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.747420 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.849444 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:52.849404 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-185.ec2.internal\" not found" Jul 10 20:34:52.867523 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.867500 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:52.942717 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.942684 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" Jul 10 20:34:52.954683 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.954656 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:34:52.956691 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.956676 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" Jul 10 20:34:52.965289 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.965271 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 20:34:52.974019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:52.974002 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:53.423135 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.423099 2566 apiserver.go:52] "Watching apiserver" Jul 10 20:34:53.433480 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.433379 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 20:34:53.434590 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.434562 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2dfsf","kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz","openshift-cluster-node-tuning-operator/tuned-gndf7","openshift-dns/node-resolver-hfcmm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal","openshift-multus/multus-vqbfr","openshift-image-registry/node-ca-wlsl8","openshift-multus/multus-additional-cni-plugins-jf7ht","openshift-multus/network-metrics-daemon-w5tk4","openshift-network-diagnostics/network-check-target-spzdr","openshift-network-operator/iptables-alerter-z5cst","openshift-ovn-kubernetes/ovnkube-node-2577q","kube-system/global-pull-secret-syncer-4lx9r"] Jul 10 20:34:53.437266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.437239 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.439708 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.439592 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 20:34:53.439708 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.439623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.439960 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.439943 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.440346 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.440327 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 20:34:53.440440 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.440405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lchzf\"" Jul 10 20:34:53.440530 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.440512 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.441906 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.441868 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.442399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442217 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.442399 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442267 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6vcb9\"" Jul 10 20:34:53.442548 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442470 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 20:34:53.442756 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442731 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 20:34:53.442911 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442737 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.442988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.442961 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 20:34:53.443269 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.443244 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 20:34:53.443350 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.443334 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 20:34:53.444183 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.444162 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.445286 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.444902 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.445286 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.445064 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.445286 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.445218 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 20:34:53.445488 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.445323 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rfspx\"" Jul 10 20:34:53.447270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.447099 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.447270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.447099 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.447644 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.447543 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.447733 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.447712 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4mwhm\"" Jul 10 20:34:53.449622 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.449605 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.450083 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.450058 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f57gp\"" Jul 10 20:34:53.450083 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.450079 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.450251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.450065 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.450315 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.450258 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:34:53.453004 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.452982 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-bin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-systemd\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-etc-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453322 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-os-release\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-hostroot\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-systemd-units\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-multus\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-etc-kubernetes\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-ovn\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-tmp\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.453570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cnibin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453757 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-system-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cni-binary-copy\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-socket-dir-parent\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-node-log\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453968 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-sys\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.453993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-var-lib-kubelet\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-netd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-script-lib\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysconfig\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-kubernetes\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.454251 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-k8s-cni-cncf-io\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454246 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-netns\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-daemon-config\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454297 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-config\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-conf\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-log-socket\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-bin\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b2tk\" (UniqueName: \"kubernetes.io/projected/f4a442e0-6961-480d-a8a8-acfc561d7457-kube-api-access-5b2tk\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-env-overrides\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-registration-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-conf-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-var-lib-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-host\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454599 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-etc-tuned\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjt9l\" (UniqueName: \"kubernetes.io/projected/3510ad80-d67c-4e36-af4b-82724ce56833-kube-api-access-zjt9l\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-kubelet\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-multus-certs\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-slash\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-netns\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-systemd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-device-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-sys-fs\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-modprobe-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.454985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovn-node-metrics-cert\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455054 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9kbq\" (UniqueName: \"kubernetes.io/projected/cb9a1dcd-09c5-4a70-9fbf-57616552087a-kube-api-access-f9kbq\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-run\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqptw\" (UniqueName: \"kubernetes.io/projected/b54b2f36-4c49-40fe-9fcc-36c74d12a977-kube-api-access-mqptw\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-kubelet\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-socket-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455255 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-lib-modules\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455424 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-26bht\"" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455565 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 20:34:53.455941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.455768 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 20:34:53.456837 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.456219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.458921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.458885 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.460698 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.460249 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bnxbw\"" Jul 10 20:34:53.462441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.460822 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 20:34:53.462441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.461895 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.462441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.462142 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 20:34:53.462664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.462639 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.464415 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.462943 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 20:34:53.464415 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.464276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qqjrn\"" Jul 10 20:34:53.466225 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.466150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.466311 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.466231 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:34:53.468687 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.468668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:53.468789 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.468747 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:34:53.471241 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.471180 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.473392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.473369 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 20:34:53.473734 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.473710 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:34:53.473844 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.473819 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 20:34:53.473933 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.473721 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l8kjk\"" Jul 10 20:34:53.477474 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.477450 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r42cq" Jul 10 20:34:53.485334 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.485313 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r42cq" Jul 10 20:34:53.544885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.544863 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 20:34:53.556106 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556114 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovn-node-metrics-cert\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556134 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9kbq\" (UniqueName: \"kubernetes.io/projected/cb9a1dcd-09c5-4a70-9fbf-57616552087a-kube-api-access-f9kbq\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-run\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dg9l\" (UniqueName: \"kubernetes.io/projected/04669449-ce1e-40ad-804b-7b1bb3f744c6-kube-api-access-2dg9l\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7ggs\" (UniqueName: \"kubernetes.io/projected/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-kube-api-access-c7ggs\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556242 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a341f91e-55e6-42a5-b430-2b2c3b2827e9-hosts-file\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556263 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a341f91e-55e6-42a5-b430-2b2c3b2827e9-tmp-dir\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556289 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqptw\" (UniqueName: \"kubernetes.io/projected/b54b2f36-4c49-40fe-9fcc-36c74d12a977-kube-api-access-mqptw\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.556321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-kubelet\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-socket-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556351 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-run\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556364 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-lib-modules\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfh7h\" (UniqueName: \"kubernetes.io/projected/a341f91e-55e6-42a5-b430-2b2c3b2827e9-kube-api-access-bfh7h\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556438 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/907d43a7-938d-4b6b-bc24-27079a0ad349-konnectivity-ca\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-bin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556487 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-socket-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-systemd\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-kubelet\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-host\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556549 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-bin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556553 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-serviceca\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556571 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-etc-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.556762 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556601 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-lib-modules\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556604 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-systemd\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-os-release\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-etc-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-hostroot\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556643 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-hostroot\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-systemd-units\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-os-release\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-os-release\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556805 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-systemd-units\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-kubelet-config\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-multus\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-etc-kubernetes\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-ovn\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556971 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-etc-kubernetes\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/907d43a7-938d-4b6b-bc24-27079a0ad349-agent-certs\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.557536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.556973 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-cni-multus\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-tmp\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-ovn\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-system-cni-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-binary-copy\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-iptables-alerter-script\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557132 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-host-slash\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cnibin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-system-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cnibin\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557321 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7mjv\" (UniqueName: \"kubernetes.io/projected/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-kube-api-access-c7mjv\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.558501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-system-cni-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cni-binary-copy\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-socket-dir-parent\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-node-log\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-sys\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-socket-dir-parent\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-var-lib-kubelet\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-var-lib-kubelet\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557563 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557621 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-netd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557677 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-etc-selinux\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-script-lib\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysconfig\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557729 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-sys\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-kubernetes\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.559405 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557768 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-netd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-dbus\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557819 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-cni-binary-copy\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557841 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysconfig\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-k8s-cni-cncf-io\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-netns\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557891 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-kubernetes\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557927 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-netns\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557806 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-node-log\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-daemon-config\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.557974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-config\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-conf\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558068 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558096 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-log-socket\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.560299 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-bin\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b2tk\" (UniqueName: \"kubernetes.io/projected/f4a442e0-6961-480d-a8a8-acfc561d7457-kube-api-access-5b2tk\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558184 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-env-overrides\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558323 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-sysctl-conf\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558338 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-daemon-config\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-registration-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cnibin\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558400 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-log-socket\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-conf-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-var-lib-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558520 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-script-lib\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-host\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-k8s-cni-cncf-io\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-host\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558604 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-registration-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558613 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-multus-conf-dir\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558627 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-var-lib-openvswitch\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.561177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-etc-tuned\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-cni-bin\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjt9l\" (UniqueName: \"kubernetes.io/projected/3510ad80-d67c-4e36-af4b-82724ce56833-kube-api-access-zjt9l\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558771 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovnkube-config\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt95r\" (UniqueName: \"kubernetes.io/projected/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-kube-api-access-tt95r\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.558944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-kubelet\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-multus-certs\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-slash\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-var-lib-kubelet\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b54b2f36-4c49-40fe-9fcc-36c74d12a977-host-run-multus-certs\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-netns\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-systemd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559139 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-slash\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-device-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559168 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-host-run-netns\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb9a1dcd-09c5-4a70-9fbf-57616552087a-env-overrides\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559194 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb9a1dcd-09c5-4a70-9fbf-57616552087a-run-systemd\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.562037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-device-dir\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.562825 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-sys-fs\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.562825 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559663 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4a442e0-6961-480d-a8a8-acfc561d7457-sys-fs\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.562825 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.559733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-modprobe-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.562825 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.560158 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3510ad80-d67c-4e36-af4b-82724ce56833-etc-modprobe-d\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.564189 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.564166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-tmp\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.564359 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.564228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3510ad80-d67c-4e36-af4b-82724ce56833-etc-tuned\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.564801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.564781 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb9a1dcd-09c5-4a70-9fbf-57616552087a-ovn-node-metrics-cert\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.568250 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.568225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b2tk\" (UniqueName: \"kubernetes.io/projected/f4a442e0-6961-480d-a8a8-acfc561d7457-kube-api-access-5b2tk\") pod \"aws-ebs-csi-driver-node-hf8mz\" (UID: \"f4a442e0-6961-480d-a8a8-acfc561d7457\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.568582 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.568559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqptw\" (UniqueName: \"kubernetes.io/projected/b54b2f36-4c49-40fe-9fcc-36c74d12a977-kube-api-access-mqptw\") pod \"multus-vqbfr\" (UID: \"b54b2f36-4c49-40fe-9fcc-36c74d12a977\") " pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.569348 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.569331 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjt9l\" (UniqueName: \"kubernetes.io/projected/3510ad80-d67c-4e36-af4b-82724ce56833-kube-api-access-zjt9l\") pod \"tuned-gndf7\" (UID: \"3510ad80-d67c-4e36-af4b-82724ce56833\") " pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.570178 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.570158 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9kbq\" (UniqueName: \"kubernetes.io/projected/cb9a1dcd-09c5-4a70-9fbf-57616552087a-kube-api-access-f9kbq\") pod \"ovnkube-node-2577q\" (UID: \"cb9a1dcd-09c5-4a70-9fbf-57616552087a\") " pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.594992 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.594944 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"b338db47c4937c088ae739177bb426e7","Type":"ContainerStarted","Data":"b1cea374efda3c4f4552624f3eb67c54a6f9a217fe65d0bdec0740b0e4ea8d51"} Jul 10 20:34:53.596224 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.596201 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerStarted","Data":"abffd69d8c93948a35c79748737f1d17d57d728bf5846cdd529851e51578787e"} Jul 10 20:34:53.654954 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.654919 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:53.661091 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661063 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a341f91e-55e6-42a5-b430-2b2c3b2827e9-hosts-file\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a341f91e-55e6-42a5-b430-2b2c3b2827e9-tmp-dir\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfh7h\" (UniqueName: \"kubernetes.io/projected/a341f91e-55e6-42a5-b430-2b2c3b2827e9-kube-api-access-bfh7h\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/907d43a7-938d-4b6b-bc24-27079a0ad349-konnectivity-ca\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a341f91e-55e6-42a5-b430-2b2c3b2827e9-hosts-file\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661193 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-host\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.661243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-serviceca\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.661534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-host\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.661534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661421 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-os-release\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-kubelet-config\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.661534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/907d43a7-938d-4b6b-bc24-27079a0ad349-agent-certs\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.661534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-system-cni-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-binary-copy\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-os-release\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-iptables-alerter-script\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-host-slash\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661689 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-serviceca\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7mjv\" (UniqueName: \"kubernetes.io/projected/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-kube-api-access-c7mjv\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:53.661791 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-dbus\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cnibin\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt95r\" (UniqueName: \"kubernetes.io/projected/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-kube-api-access-tt95r\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.661978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dg9l\" (UniqueName: \"kubernetes.io/projected/04669449-ce1e-40ad-804b-7b1bb3f744c6-kube-api-access-2dg9l\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7ggs\" (UniqueName: \"kubernetes.io/projected/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-kube-api-access-c7ggs\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-kubelet-config\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-host-slash\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-iptables-alerter-script\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.662290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-system-cni-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662314 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cnibin\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/092b3231-427e-4ba4-99a2-ca436e68be3b-dbus\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662386 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a341f91e-55e6-42a5-b430-2b2c3b2827e9-tmp-dir\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.662410 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.662488 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:34:54.162465516 +0000 UTC m=+3.046296840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662554 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.662413 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.662808 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:34:54.162792264 +0000 UTC m=+3.046623587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:53.662968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.662903 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/907d43a7-938d-4b6b-bc24-27079a0ad349-konnectivity-ca\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.663424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.663037 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-cni-binary-copy\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.663424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.663239 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.664871 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.664842 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/907d43a7-938d-4b6b-bc24-27079a0ad349-agent-certs\") pod \"konnectivity-agent-2dfsf\" (UID: \"907d43a7-938d-4b6b-bc24-27079a0ad349\") " pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.669430 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.669296 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:34:53.669430 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.669320 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:34:53.669430 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.669335 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:53.669430 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:53.669401 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:34:54.169384623 +0000 UTC m=+3.053215963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:53.669720 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.669429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfh7h\" (UniqueName: \"kubernetes.io/projected/a341f91e-55e6-42a5-b430-2b2c3b2827e9-kube-api-access-bfh7h\") pod \"node-resolver-hfcmm\" (UID: \"a341f91e-55e6-42a5-b430-2b2c3b2827e9\") " pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.671167 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.671143 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7ggs\" (UniqueName: \"kubernetes.io/projected/6e9bd4e0-daaf-498d-864a-3b0904b69fb7-kube-api-access-c7ggs\") pod \"iptables-alerter-z5cst\" (UID: \"6e9bd4e0-daaf-498d-864a-3b0904b69fb7\") " pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.671437 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.671411 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt95r\" (UniqueName: \"kubernetes.io/projected/1d1408fb-0169-487a-ac62-3cdf58ef7c1d-kube-api-access-tt95r\") pod \"node-ca-wlsl8\" (UID: \"1d1408fb-0169-487a-ac62-3cdf58ef7c1d\") " pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:53.671900 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.671865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7mjv\" (UniqueName: \"kubernetes.io/projected/979fdca6-b1a1-4f8e-bdf6-17f49a4aff69-kube-api-access-c7mjv\") pod \"multus-additional-cni-plugins-jf7ht\" (UID: \"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69\") " pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.672182 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.672166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dg9l\" (UniqueName: \"kubernetes.io/projected/04669449-ce1e-40ad-804b-7b1bb3f744c6-kube-api-access-2dg9l\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:53.753225 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.753136 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vqbfr" Jul 10 20:34:53.762075 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.762051 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:34:53.771998 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.771975 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" Jul 10 20:34:53.784925 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.784905 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gndf7" Jul 10 20:34:53.794808 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.794789 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z5cst" Jul 10 20:34:53.803376 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.803356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hfcmm" Jul 10 20:34:53.811755 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.811726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" Jul 10 20:34:53.819639 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.819619 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:34:53.820921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.820898 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 20:34:53.825050 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:53.825032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wlsl8" Jul 10 20:34:54.166140 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.166056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:54.166140 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.166104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:54.166391 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.166249 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:54.166391 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.166314 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:34:55.166298061 +0000 UTC m=+4.050129387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:54.166391 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.166249 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:54.166507 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.166470 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:34:55.166449387 +0000 UTC m=+4.050280715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:54.266971 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.266931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:54.267140 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.267107 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:34:54.267140 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.267128 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:34:54.267140 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.267138 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:54.267265 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.267194 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:34:55.267179177 +0000 UTC m=+4.151010500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:54.488942 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.488827 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:29:54 +0000 UTC" deadline="2028-02-01 20:34:46.892759008 +0000 UTC" Jul 10 20:34:54.488942 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.488887 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13703h59m52.403877s" Jul 10 20:34:54.591562 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:54.591526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:54.591756 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:54.591673 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:34:54.689183 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.689139 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979fdca6_b1a1_4f8e_bdf6_17f49a4aff69.slice/crio-77c3e7a8d893d060e4395ec12d0fe9fba4d6ef913961649300c6bbf17252e956 WatchSource:0}: Error finding container 77c3e7a8d893d060e4395ec12d0fe9fba4d6ef913961649300c6bbf17252e956: Status 404 returned error can't find the container with id 77c3e7a8d893d060e4395ec12d0fe9fba4d6ef913961649300c6bbf17252e956 Jul 10 20:34:54.692829 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.692801 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1408fb_0169_487a_ac62_3cdf58ef7c1d.slice/crio-02ee7f5c813d8da695e1f382b5eacc88003355768cf4469d99144abcbd9c27fa WatchSource:0}: Error finding container 02ee7f5c813d8da695e1f382b5eacc88003355768cf4469d99144abcbd9c27fa: Status 404 returned error can't find the container with id 02ee7f5c813d8da695e1f382b5eacc88003355768cf4469d99144abcbd9c27fa Jul 10 20:34:54.696406 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.696383 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9bd4e0_daaf_498d_864a_3b0904b69fb7.slice/crio-772c6e286cbdab56a72f0a3244b1be6bfab2c871d719ba5ea01d93919acff127 WatchSource:0}: Error finding container 772c6e286cbdab56a72f0a3244b1be6bfab2c871d719ba5ea01d93919acff127: Status 404 returned error can't find the container with id 772c6e286cbdab56a72f0a3244b1be6bfab2c871d719ba5ea01d93919acff127 Jul 10 20:34:54.697468 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.697332 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54b2f36_4c49_40fe_9fcc_36c74d12a977.slice/crio-2acf6742dbd288496e9a6e6fada11b230d80da126cc63b1bac093ea1d19d8bba WatchSource:0}: Error finding container 2acf6742dbd288496e9a6e6fada11b230d80da126cc63b1bac093ea1d19d8bba: Status 404 returned error can't find the container with id 2acf6742dbd288496e9a6e6fada11b230d80da126cc63b1bac093ea1d19d8bba Jul 10 20:34:54.698366 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.698288 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4a442e0_6961_480d_a8a8_acfc561d7457.slice/crio-b595c19303774b6751045f08f363f5fc2d1c9ee1b8a061ba4b12bdcaa82d43bb WatchSource:0}: Error finding container b595c19303774b6751045f08f363f5fc2d1c9ee1b8a061ba4b12bdcaa82d43bb: Status 404 returned error can't find the container with id b595c19303774b6751045f08f363f5fc2d1c9ee1b8a061ba4b12bdcaa82d43bb Jul 10 20:34:54.699183 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.699006 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9a1dcd_09c5_4a70_9fbf_57616552087a.slice/crio-f10c2ed2e12f1fa6e050cace74182d716a606bb65554093ae9272fd6bed632d7 WatchSource:0}: Error finding container f10c2ed2e12f1fa6e050cace74182d716a606bb65554093ae9272fd6bed632d7: Status 404 returned error can't find the container with id f10c2ed2e12f1fa6e050cace74182d716a606bb65554093ae9272fd6bed632d7 Jul 10 20:34:54.701758 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.701720 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod907d43a7_938d_4b6b_bc24_27079a0ad349.slice/crio-377a4149c28c187939022db3691373d9cae8e15afadfd175760c6a4e17323798 WatchSource:0}: Error finding container 377a4149c28c187939022db3691373d9cae8e15afadfd175760c6a4e17323798: Status 404 returned error can't find the container with id 377a4149c28c187939022db3691373d9cae8e15afadfd175760c6a4e17323798 Jul 10 20:34:54.702268 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.702247 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3510ad80_d67c_4e36_af4b_82724ce56833.slice/crio-fdb34267724d5ab5468b11f4b32d1edff338e7310bfa73ae1706fbf084201de3 WatchSource:0}: Error finding container fdb34267724d5ab5468b11f4b32d1edff338e7310bfa73ae1706fbf084201de3: Status 404 returned error can't find the container with id fdb34267724d5ab5468b11f4b32d1edff338e7310bfa73ae1706fbf084201de3 Jul 10 20:34:54.703406 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:34:54.703384 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda341f91e_55e6_42a5_b430_2b2c3b2827e9.slice/crio-ab082756dab6174a4c1115a6eed10b6877effab87f23e70d8478e04c792f39c9 WatchSource:0}: Error finding container ab082756dab6174a4c1115a6eed10b6877effab87f23e70d8478e04c792f39c9: Status 404 returned error can't find the container with id ab082756dab6174a4c1115a6eed10b6877effab87f23e70d8478e04c792f39c9 Jul 10 20:34:55.174815 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.174472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:55.174815 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.174769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:55.175130 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.174658 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:55.175130 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.174903 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:34:57.174879879 +0000 UTC m=+6.058711222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:55.175130 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.174947 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:55.175130 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.175003 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:34:57.174986699 +0000 UTC m=+6.058818038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:55.275699 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.275657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:55.275941 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.275923 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:34:55.276045 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.275949 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:34:55.276045 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.275963 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:55.276045 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.276025 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:34:57.276004259 +0000 UTC m=+6.159835597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:55.489339 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.488991 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:29:54 +0000 UTC" deadline="2027-12-13 00:13:43.821631279 +0000 UTC" Jul 10 20:34:55.489339 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.489059 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12483h38m48.33257696s" Jul 10 20:34:55.591951 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.591403 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:55.591951 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.591540 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:34:55.592472 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.592295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:55.592472 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:55.592423 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:34:55.622475 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.622429 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" event={"ID":"b338db47c4937c088ae739177bb426e7","Type":"ContainerStarted","Data":"b58f71bc2f4e56614ac2c5ede30be73a93992486d81738e43c707bb13668bf3c"} Jul 10 20:34:55.633370 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.633325 2566 generic.go:356] "Generic (PLEG): container finished" podID="29da48fefeff7a1251df734706134371" containerID="4f8b69220b8af3e5e438e778e2a356b2bc3fdc8b95429c2d7abd9fc977e35f61" exitCode=0 Jul 10 20:34:55.633509 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.633423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerDied","Data":"4f8b69220b8af3e5e438e778e2a356b2bc3fdc8b95429c2d7abd9fc977e35f61"} Jul 10 20:34:55.648569 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.648531 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2dfsf" event={"ID":"907d43a7-938d-4b6b-bc24-27079a0ad349","Type":"ContainerStarted","Data":"377a4149c28c187939022db3691373d9cae8e15afadfd175760c6a4e17323798"} Jul 10 20:34:55.655504 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.655451 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-185.ec2.internal" podStartSLOduration=2.655431835 podStartE2EDuration="2.655431835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:34:55.639551339 +0000 UTC m=+4.523382686" watchObservedRunningTime="2026-07-10 20:34:55.655431835 +0000 UTC m=+4.539263184" Jul 10 20:34:55.670069 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.670038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"f10c2ed2e12f1fa6e050cace74182d716a606bb65554093ae9272fd6bed632d7"} Jul 10 20:34:55.673513 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.673478 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" event={"ID":"f4a442e0-6961-480d-a8a8-acfc561d7457","Type":"ContainerStarted","Data":"b595c19303774b6751045f08f363f5fc2d1c9ee1b8a061ba4b12bdcaa82d43bb"} Jul 10 20:34:55.698369 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.698033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wlsl8" event={"ID":"1d1408fb-0169-487a-ac62-3cdf58ef7c1d","Type":"ContainerStarted","Data":"02ee7f5c813d8da695e1f382b5eacc88003355768cf4469d99144abcbd9c27fa"} Jul 10 20:34:55.701945 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.701908 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hfcmm" event={"ID":"a341f91e-55e6-42a5-b430-2b2c3b2827e9","Type":"ContainerStarted","Data":"ab082756dab6174a4c1115a6eed10b6877effab87f23e70d8478e04c792f39c9"} Jul 10 20:34:55.714742 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.714703 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gndf7" event={"ID":"3510ad80-d67c-4e36-af4b-82724ce56833","Type":"ContainerStarted","Data":"fdb34267724d5ab5468b11f4b32d1edff338e7310bfa73ae1706fbf084201de3"} Jul 10 20:34:55.723603 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.723573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vqbfr" event={"ID":"b54b2f36-4c49-40fe-9fcc-36c74d12a977","Type":"ContainerStarted","Data":"2acf6742dbd288496e9a6e6fada11b230d80da126cc63b1bac093ea1d19d8bba"} Jul 10 20:34:55.726869 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.726804 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z5cst" event={"ID":"6e9bd4e0-daaf-498d-864a-3b0904b69fb7","Type":"ContainerStarted","Data":"772c6e286cbdab56a72f0a3244b1be6bfab2c871d719ba5ea01d93919acff127"} Jul 10 20:34:55.744904 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:55.740757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerStarted","Data":"77c3e7a8d893d060e4395ec12d0fe9fba4d6ef913961649300c6bbf17252e956"} Jul 10 20:34:56.591171 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:56.591138 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:56.591646 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:56.591275 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:34:56.759895 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:56.759790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" event={"ID":"29da48fefeff7a1251df734706134371","Type":"ContainerStarted","Data":"c0399e020b4c4c0c4132bb0a7573892cc6e8cd1082da5ab70d289de3cf204f1f"} Jul 10 20:34:57.191557 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:57.191516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:57.191752 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:57.191574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:57.191752 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.191660 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:57.191752 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.191687 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:57.191974 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.191755 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:35:01.191734894 +0000 UTC m=+10.075566224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:34:57.191974 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.191777 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:01.191766625 +0000 UTC m=+10.075597953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:34:57.292701 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:57.292658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:57.292910 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.292873 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:34:57.292910 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.292894 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:34:57.292910 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.292908 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:57.293093 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.292969 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:35:01.292949775 +0000 UTC m=+10.176781101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:34:57.592594 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:57.592013 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:57.592594 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.592149 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:34:57.593848 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:57.593441 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:57.593848 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:57.593570 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:34:58.591021 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:58.590983 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:34:58.591203 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:58.591121 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:34:59.591486 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:59.591453 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:34:59.591973 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:59.591622 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:34:59.591973 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:34:59.591673 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:34:59.591973 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:34:59.591807 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:00.590520 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:00.590479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:00.590704 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:00.590624 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:01.230181 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:01.230238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.230369 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.230437 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:35:09.230416984 +0000 UTC m=+18.114248309 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.230842 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:01.230947 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.230910 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:09.230876495 +0000 UTC m=+18.114707821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:01.331344 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:01.331292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:01.331556 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.331540 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:35:01.331636 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.331561 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:35:01.331636 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.331574 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:01.331735 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.331638 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:35:09.331617537 +0000 UTC m=+18.215448863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:01.592540 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:01.592386 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:01.592540 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.592510 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:01.592777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:01.592627 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:01.592777 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:01.592753 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:02.591395 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:02.590868 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:02.591395 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:02.591014 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:03.591682 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:03.591599 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:03.592174 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:03.591731 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:03.592174 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:03.591997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:03.592174 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:03.592119 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:04.591354 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:04.591322 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:04.591544 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:04.591447 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:05.591223 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:05.591181 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:05.591680 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:05.591328 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:05.591680 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:05.591377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:05.591680 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:05.591487 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:06.590531 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:06.590449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:06.590685 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:06.590601 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:07.590986 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:07.590939 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:07.591447 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:07.590941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:07.591447 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:07.591069 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:07.591447 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:07.591185 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:08.591366 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:08.591332 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:08.591901 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:08.591455 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:09.294991 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:09.294945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:09.295162 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:09.295055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:09.295162 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.295110 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:09.295162 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.295154 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:09.295308 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.295187 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:35:25.295162844 +0000 UTC m=+34.178994182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:09.295308 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.295206 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:25.295196264 +0000 UTC m=+34.179027588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:09.395928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:09.395889 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:09.396132 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.396099 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:35:09.396132 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.396128 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:35:09.396252 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.396144 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:09.396252 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.396213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:35:25.396193089 +0000 UTC m=+34.280024422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:09.590831 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:09.590742 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:09.591041 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.590941 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:09.591041 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:09.591021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:09.591144 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:09.591125 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:10.590409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:10.590374 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:10.590960 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:10.590498 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:11.591516 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:11.591479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:11.591962 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:11.591599 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:11.591962 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:11.591647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:11.591962 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:11.591797 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:12.591437 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:12.591398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:12.591642 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:12.591523 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:13.591420 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:13.591372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:13.591420 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:13.591428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:13.592078 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:13.591525 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:13.592078 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:13.591659 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:14.590629 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.590358 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:14.590762 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:14.590724 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:14.807701 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.807666 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hfcmm" event={"ID":"a341f91e-55e6-42a5-b430-2b2c3b2827e9","Type":"ContainerStarted","Data":"a19be436321d4d63bfa3c006c45d56dc2321f38c63be04d739b292e14f08e04f"} Jul 10 20:35:14.809274 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.809248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gndf7" event={"ID":"3510ad80-d67c-4e36-af4b-82724ce56833","Type":"ContainerStarted","Data":"a1b783c2028e8aa7efc6d84ea8ee4f342e671b5f3607c634f66a73c8a570ec07"} Jul 10 20:35:14.810831 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.810793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vqbfr" event={"ID":"b54b2f36-4c49-40fe-9fcc-36c74d12a977","Type":"ContainerStarted","Data":"5fab05ca4d5793d01fdafee799965ab86bdbbcf6876aaaaffa568c54b9edf254"} Jul 10 20:35:14.812231 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.812208 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="55b2524a858283b5affb3a8f0cc3fc5fd8e3bb8c4502f953e3b64c11439ba5a9" exitCode=0 Jul 10 20:35:14.812330 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.812287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"55b2524a858283b5affb3a8f0cc3fc5fd8e3bb8c4502f953e3b64c11439ba5a9"} Jul 10 20:35:14.813766 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.813732 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2dfsf" event={"ID":"907d43a7-938d-4b6b-bc24-27079a0ad349","Type":"ContainerStarted","Data":"dd26fc9cb38ed9d4612b889088e949a8582894a366b9c33eee4ab781b59d65a8"} Jul 10 20:35:14.816424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816407 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:35:14.816717 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816698 2566 generic.go:356] "Generic (PLEG): container finished" podID="cb9a1dcd-09c5-4a70-9fbf-57616552087a" containerID="e1d6d0e7f09da43dca01505b25efa4447870d47f60b6f2a6c97f467a2091819e" exitCode=1 Jul 10 20:35:14.816802 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"56d002bd20ba78475c950bd45046b676350d9acc5f5b126a8333382b4ebf5c5a"} Jul 10 20:35:14.816868 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"6f31f3c7d5ae50592ffe1c44105308801f17e8c9a42d5adae499c8da8c9aa9bd"} Jul 10 20:35:14.816868 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816862 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"62cd601bbe9571dd373ddee8e62d2e74024e0ed2b4893e23d00cdf9962ff19e1"} Jul 10 20:35:14.816941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816913 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"a59856fdf33b1b4dafc1880545359bbee41bea4b760a522c9131ad485a7f1268"} Jul 10 20:35:14.816974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816945 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerDied","Data":"e1d6d0e7f09da43dca01505b25efa4447870d47f60b6f2a6c97f467a2091819e"} Jul 10 20:35:14.817022 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.816989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"8099cbc52388fdf96506c15300ac283aa4ca31afc3c9c11a8006fb634d67b169"} Jul 10 20:35:14.819620 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.819215 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" event={"ID":"f4a442e0-6961-480d-a8a8-acfc561d7457","Type":"ContainerStarted","Data":"fc21a3c34ef6ba235a3663e0d74766c021097ebf53a8570e4ae768b63649fe4c"} Jul 10 20:35:14.820924 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.820888 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-185.ec2.internal" podStartSLOduration=21.820854609 podStartE2EDuration="21.820854609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:34:56.776295586 +0000 UTC m=+5.660126934" watchObservedRunningTime="2026-07-10 20:35:14.820854609 +0000 UTC m=+23.704685954" Jul 10 20:35:14.821063 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.821043 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wlsl8" event={"ID":"1d1408fb-0169-487a-ac62-3cdf58ef7c1d","Type":"ContainerStarted","Data":"809c8977a5e4992c819506ea465ac44c159aab590e4f5272593452f2980891cc"} Jul 10 20:35:14.821693 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.821664 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hfcmm" podStartSLOduration=3.417446903 podStartE2EDuration="22.821653402s" totalImagesPullingTime="19.404206499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:14.820989962 +0000 UTC m=+23.704821290" watchObservedRunningTime="2026-07-10 20:35:14.821653402 +0000 UTC m=+23.705484745" Jul 10 20:35:14.835604 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.835561 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gndf7" podStartSLOduration=3.3976721850000002 podStartE2EDuration="22.835550838s" totalImagesPullingTime="19.437878653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:14.835028301 +0000 UTC m=+23.718859645" watchObservedRunningTime="2026-07-10 20:35:14.835550838 +0000 UTC m=+23.719382181" Jul 10 20:35:14.849093 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.849059 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vqbfr" podStartSLOduration=3.361298121 podStartE2EDuration="22.849047843s" totalImagesPullingTime="19.487749722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:14.848598764 +0000 UTC m=+23.732430107" watchObservedRunningTime="2026-07-10 20:35:14.849047843 +0000 UTC m=+23.732879187" Jul 10 20:35:14.878960 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.878872 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2dfsf" podStartSLOduration=3.5001378499999998 podStartE2EDuration="22.87885719s" totalImagesPullingTime="19.37871934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:14.878839122 +0000 UTC m=+23.762670459" watchObservedRunningTime="2026-07-10 20:35:14.87885719 +0000 UTC m=+23.762688530" Jul 10 20:35:14.892307 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.892266 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wlsl8" podStartSLOduration=3.457726715 podStartE2EDuration="22.892254815s" totalImagesPullingTime="19.4345281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:14.891912517 +0000 UTC m=+23.775743859" watchObservedRunningTime="2026-07-10 20:35:14.892254815 +0000 UTC m=+23.776086158" Jul 10 20:35:14.991008 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:14.990976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:15.067759 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.067722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:15.119227 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.119187 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:15.590516 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.590445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:15.590690 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:15.590575 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:15.590690 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.590629 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:15.590793 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:15.590746 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:15.824847 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.824795 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z5cst" event={"ID":"6e9bd4e0-daaf-498d-864a-3b0904b69fb7","Type":"ContainerStarted","Data":"0f3aab293b7da4c29f04fbdb0c9a792556660b7cb04675fedb9624670bb322d3"} Jul 10 20:35:15.839309 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.839256 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z5cst" podStartSLOduration=4.402174729 podStartE2EDuration="23.83923835s" totalImagesPullingTime="19.437063621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:15.838695425 +0000 UTC m=+24.722526770" watchObservedRunningTime="2026-07-10 20:35:15.83923835 +0000 UTC m=+24.723069694" Jul 10 20:35:15.955859 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:15.955819 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:16.010792 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.010758 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:16.028509 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.028483 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 20:35:16.514278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.514171 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T20:35:16.028501194Z","UUID":"bb61e253-8329-49d2-a88f-6d10eec68511","Handler":null,"Name":"","Endpoint":""} Jul 10 20:35:16.515904 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.515881 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 20:35:16.516032 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.515927 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 20:35:16.590608 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.590578 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:16.590799 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:16.590718 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:16.829573 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:16.829487 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" event={"ID":"f4a442e0-6961-480d-a8a8-acfc561d7457","Type":"ContainerStarted","Data":"b2dd84fb3dba8498bf8eaff8e8586bf0c691bb40237ebf3844442adeeaabf6c6"} Jul 10 20:35:17.591036 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:17.591001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:17.591229 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:17.591138 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:17.591229 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:17.591197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:17.591354 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:17.591331 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:17.835300 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:17.835116 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:35:17.835717 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:17.835635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"60755bb51572c657cc6d8ee15c62ea29a01de02b8d187d2dea326b6e1e7cbeb8"} Jul 10 20:35:18.590939 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:18.590904 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:18.591108 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:18.591043 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:18.965592 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:18.965564 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:35:18.966276 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:18.966255 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:35:19.590997 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.590802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:19.591149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.590802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:19.591149 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:19.591085 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:19.591149 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:19.591138 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:19.843197 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.843130 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:35:19.843531 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.843510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"60367ee0ea067e83a4790ac70c6f6f72b5d9015229277703d6d45d3ce3315260"} Jul 10 20:35:19.843766 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.843741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:19.843953 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.843938 2566 scope.go:117] "RemoveContainer" containerID="e1d6d0e7f09da43dca01505b25efa4447870d47f60b6f2a6c97f467a2091819e" Jul 10 20:35:19.845771 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.845745 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" event={"ID":"f4a442e0-6961-480d-a8a8-acfc561d7457","Type":"ContainerStarted","Data":"f2e382dbd3268b591cc96c76b7d1b08cd86147f2bb661b4d1092fbfa0669adf1"} Jul 10 20:35:19.847638 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.847612 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="9dd1ab61b86a38d1f281a656b7a6e517a67795f118cbebc5ff997e52f57cbd76" exitCode=0 Jul 10 20:35:19.847736 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.847701 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"9dd1ab61b86a38d1f281a656b7a6e517a67795f118cbebc5ff997e52f57cbd76"} Jul 10 20:35:19.847819 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.847791 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:35:19.848320 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.848289 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2dfsf" Jul 10 20:35:19.860369 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.860350 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:19.908524 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.908483 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hf8mz" podStartSLOduration=4.015820586 podStartE2EDuration="27.908469371s" totalImagesPullingTime="23.892648785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:19.908346968 +0000 UTC m=+28.792178313" watchObservedRunningTime="2026-07-10 20:35:19.908469371 +0000 UTC m=+28.792300714" Jul 10 20:35:19.982162 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:19.982138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:20.591055 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.591024 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:20.591265 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:20.591129 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:20.854617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.854548 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:35:20.855007 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.854978 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" event={"ID":"cb9a1dcd-09c5-4a70-9fbf-57616552087a","Type":"ContainerStarted","Data":"5e74c8046c01088666e93773840101f421b4a62a0d1f36edd56b26c851ea2341"} Jul 10 20:35:20.855585 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.855551 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:20.855689 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.855646 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:20.873261 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.873240 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:20.882146 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.882093 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" podStartSLOduration=9.377332984 podStartE2EDuration="28.88207435s" totalImagesPullingTime="19.504741366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:20.882015019 +0000 UTC m=+29.765846363" watchObservedRunningTime="2026-07-10 20:35:20.88207435 +0000 UTC m=+29.765905686" Jul 10 20:35:20.975283 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.975249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-spzdr"] Jul 10 20:35:20.975442 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.975389 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:20.975547 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:20.975518 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:20.978185 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.978147 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lx9r"] Jul 10 20:35:20.978318 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.978289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:20.978420 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:20.978398 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:20.978837 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.978791 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w5tk4"] Jul 10 20:35:20.978941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:20.978929 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:20.979034 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:20.979013 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:21.859350 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:21.859168 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="a842d09858eaecb54f1851944d876b79b396db675c61e2347f1d58c9fe154dbb" exitCode=0 Jul 10 20:35:21.859776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:21.859256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"a842d09858eaecb54f1851944d876b79b396db675c61e2347f1d58c9fe154dbb"} Jul 10 20:35:22.591216 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:22.591186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:22.591381 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:22.591186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:22.591381 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:22.591287 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:22.591381 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:22.591186 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:22.591546 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:22.591347 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:22.591546 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:22.591470 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:22.710061 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:22.710030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:23.866036 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:23.865953 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="d43165692e4cc78f9089ae7cd89231a320c657c4892b3e78fbb689e51ecce82f" exitCode=0 Jul 10 20:35:23.866036 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:23.865997 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"d43165692e4cc78f9089ae7cd89231a320c657c4892b3e78fbb689e51ecce82f"} Jul 10 20:35:24.590789 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:24.590754 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:24.590993 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:24.590764 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:24.590993 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:24.590890 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:24.590993 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:24.590764 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:24.591191 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:24.591158 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:24.591291 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:24.591270 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:25.320653 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:25.320615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:25.320653 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:25.320665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:25.321285 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.320780 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:25.321285 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.320781 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:25.321285 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.320864 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:57.3208431 +0000 UTC m=+66.204674436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 20:35:25.321285 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.320882 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret podName:092b3231-427e-4ba4-99a2-ca436e68be3b nodeName:}" failed. No retries permitted until 2026-07-10 20:35:57.320875974 +0000 UTC m=+66.204707295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret") pod "global-pull-secret-syncer-4lx9r" (UID: "092b3231-427e-4ba4-99a2-ca436e68be3b") : object "kube-system"/"original-pull-secret" not registered Jul 10 20:35:25.421797 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:25.421759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:25.421998 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.421967 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 20:35:25.421998 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.421996 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 20:35:25.422109 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.422009 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rzkwk for pod openshift-network-diagnostics/network-check-target-spzdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:25.422109 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:25.422080 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk podName:995dfafb-2e33-4975-80f9-277f48fd8fec nodeName:}" failed. No retries permitted until 2026-07-10 20:35:57.42205528 +0000 UTC m=+66.305886616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rzkwk" (UniqueName: "kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk") pod "network-check-target-spzdr" (UID: "995dfafb-2e33-4975-80f9-277f48fd8fec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 20:35:26.590502 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.590455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:26.590502 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.590492 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:26.591149 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:26.590576 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-spzdr" podUID="995dfafb-2e33-4975-80f9-277f48fd8fec" Jul 10 20:35:26.591149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.590459 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:26.591149 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:26.590705 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lx9r" podUID="092b3231-427e-4ba4-99a2-ca436e68be3b" Jul 10 20:35:26.591149 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:26.590879 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:35:26.950916 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.950839 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-185.ec2.internal" event="NodeReady" Jul 10 20:35:26.951072 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.950994 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 20:35:26.993703 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.993651 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xtph7"] Jul 10 20:35:26.996037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.996018 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2tjkp"] Jul 10 20:35:26.996169 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.996150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:26.998054 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.998033 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:26.998441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.998236 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j2m8z\"" Jul 10 20:35:26.998441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.998302 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 20:35:26.998441 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:26.998402 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 20:35:27.000104 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.000065 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 20:35:27.000104 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.000090 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hp552\"" Jul 10 20:35:27.000323 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.000146 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 20:35:27.000379 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.000326 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 20:35:27.004278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.004251 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2tjkp"] Jul 10 20:35:27.007834 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.007576 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xtph7"] Jul 10 20:35:27.136092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hkg\" (UniqueName: \"kubernetes.io/projected/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-kube-api-access-g4hkg\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.136248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-tmp-dir\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.136248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-config-volume\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.136248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136235 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d1a662-8361-4825-b188-9d85c7f5ef94-cert\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.136393 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.136393 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.136291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99ct\" (UniqueName: \"kubernetes.io/projected/58d1a662-8361-4825-b188-9d85c7f5ef94-kube-api-access-f99ct\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.145994 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.145952 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:27.237392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-tmp-dir\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.237392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-config-volume\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.237392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d1a662-8361-4825-b188-9d85c7f5ef94-cert\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.237664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.237664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99ct\" (UniqueName: \"kubernetes.io/projected/58d1a662-8361-4825-b188-9d85c7f5ef94-kube-api-access-f99ct\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.237664 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:27.237576 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:27.237664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hkg\" (UniqueName: \"kubernetes.io/projected/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-kube-api-access-g4hkg\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.237664 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:27.237647 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:27.737625406 +0000 UTC m=+36.621456731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:27.237965 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.237940 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-config-volume\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.238728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.238711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-tmp-dir\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.242335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.242316 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58d1a662-8361-4825-b188-9d85c7f5ef94-cert\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.245138 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.245112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:27.246767 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.246744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99ct\" (UniqueName: \"kubernetes.io/projected/58d1a662-8361-4825-b188-9d85c7f5ef94-kube-api-access-f99ct\") pod \"ingress-canary-2tjkp\" (UID: \"58d1a662-8361-4825-b188-9d85c7f5ef94\") " pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.247129 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.247081 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hkg\" (UniqueName: \"kubernetes.io/projected/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-kube-api-access-g4hkg\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.316262 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.316230 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2tjkp" Jul 10 20:35:27.347396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.347360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:27.533774 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.533694 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:27.687245 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.687016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:27.743081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:27.743044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:27.743264 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:27.743222 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:27.743329 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:27.743306 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:28.74327805 +0000 UTC m=+37.627109427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:28.054796 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.054765 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2tjkp"] Jul 10 20:35:28.058101 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:35:28.058082 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d1a662_8361_4825_b188_9d85c7f5ef94.slice/crio-cae0db8bf29f2b4d743036302e967f4b958a712c03a401e00b2f932160c471af WatchSource:0}: Error finding container cae0db8bf29f2b4d743036302e967f4b958a712c03a401e00b2f932160c471af: Status 404 returned error can't find the container with id cae0db8bf29f2b4d743036302e967f4b958a712c03a401e00b2f932160c471af Jul 10 20:35:28.591321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.591230 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:28.591321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.591271 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:28.591321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.591283 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:28.593949 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.593921 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:35:28.594724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.594700 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:35:28.594840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.594724 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhdms\"" Jul 10 20:35:28.594840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.594797 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:35:28.594840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.594724 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:35:28.595008 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.594913 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rvkjn\"" Jul 10 20:35:28.749637 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.749594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:28.750197 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:28.749751 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:28.750197 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:28.749845 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:30.749826452 +0000 UTC m=+39.633657797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:28.883344 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.883257 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="0441f1949afc966c1831451ee1e1b8af3660d801d82c70cca1ce72a5a54b1839" exitCode=0 Jul 10 20:35:28.883344 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.883294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"0441f1949afc966c1831451ee1e1b8af3660d801d82c70cca1ce72a5a54b1839"} Jul 10 20:35:28.884754 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:28.884730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2tjkp" event={"ID":"58d1a662-8361-4825-b188-9d85c7f5ef94","Type":"ContainerStarted","Data":"cae0db8bf29f2b4d743036302e967f4b958a712c03a401e00b2f932160c471af"} Jul 10 20:35:29.891013 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:29.890975 2566 generic.go:356] "Generic (PLEG): container finished" podID="979fdca6-b1a1-4f8e-bdf6-17f49a4aff69" containerID="e6ac6fff1e6e1068bc58487a88cdcc5734c0a253ee707df49f584b0b7c17a517" exitCode=0 Jul 10 20:35:29.891463 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:29.891062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerDied","Data":"e6ac6fff1e6e1068bc58487a88cdcc5734c0a253ee707df49f584b0b7c17a517"} Jul 10 20:35:30.764988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:30.764948 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:30.765142 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:30.765120 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:30.765283 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:30.765188 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:34.765171133 +0000 UTC m=+43.649002454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:30.896915 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:30.896880 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" event={"ID":"979fdca6-b1a1-4f8e-bdf6-17f49a4aff69","Type":"ContainerStarted","Data":"523bb7eb24b773dde715d9174ffb17057793cab9b6f9e541e1a0391ce40862dd"} Jul 10 20:35:30.898400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:30.898367 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2tjkp" event={"ID":"58d1a662-8361-4825-b188-9d85c7f5ef94","Type":"ContainerStarted","Data":"2c0860269081b7ea6ac1235ea19f0555d585940ea5ab4f673af04951ae0b83a7"} Jul 10 20:35:30.921089 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:30.921041 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jf7ht" podStartSLOduration=8.568579968 podStartE2EDuration="38.92102545s" totalImagesPullingTime="30.352445482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:30.919558562 +0000 UTC m=+39.803389904" watchObservedRunningTime="2026-07-10 20:35:30.92102545 +0000 UTC m=+39.804856775" Jul 10 20:35:30.933322 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:30.933286 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2tjkp" podStartSLOduration=2.376165859 podStartE2EDuration="4.933272828s" totalImagesPullingTime="2.557106969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:30.932591026 +0000 UTC m=+39.816422382" watchObservedRunningTime="2026-07-10 20:35:30.933272828 +0000 UTC m=+39.817104170" Jul 10 20:35:31.005910 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:31.005824 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:31.067592 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:31.065594 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:31.117164 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:31.117134 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:31.910478 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:31.910448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:32.011100 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:32.011063 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:34.792597 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:34.792556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:34.792977 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:34.792679 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:34.792977 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:34.792732 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:42.792715947 +0000 UTC m=+51.676547269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:36.189776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:36.189745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:36.241148 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:36.241115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:42.846402 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:42.846360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:42.846923 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:42.846465 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:42.846923 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:42.846527 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:35:58.846512393 +0000 UTC m=+67.730343713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:35:49.201939 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:49.201903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:49.257666 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:49.257633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:52.883374 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:52.883346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2577q" Jul 10 20:35:53.026756 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:53.026718 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:53.089090 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:53.089057 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:35:57.347854 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.347799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:35:57.347854 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.347858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:57.350180 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.350148 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 20:35:57.350335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.350317 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 20:35:57.358940 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:57.358919 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 20:35:57.359033 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:57.358982 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:01.358961645 +0000 UTC m=+130.242792966 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : secret "metrics-daemon-secret" not found Jul 10 20:35:57.362565 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.362544 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/092b3231-427e-4ba4-99a2-ca436e68be3b-original-pull-secret\") pod \"global-pull-secret-syncer-4lx9r\" (UID: \"092b3231-427e-4ba4-99a2-ca436e68be3b\") " pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:57.417285 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.417259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lx9r" Jul 10 20:35:57.448425 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.448397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:57.450927 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.450901 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 20:35:57.461653 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.461094 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 20:35:57.471318 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.471292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzkwk\" (UniqueName: \"kubernetes.io/projected/995dfafb-2e33-4975-80f9-277f48fd8fec-kube-api-access-rzkwk\") pod \"network-check-target-spzdr\" (UID: \"995dfafb-2e33-4975-80f9-277f48fd8fec\") " pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:57.531034 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.530957 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lx9r"] Jul 10 20:35:57.535355 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:35:57.535325 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092b3231_427e_4ba4_99a2_ca436e68be3b.slice/crio-aa49fca7ff6ceef8b9e14b8827c804d5f1bbbc68773aab1a7eabc4c04fd0e34a WatchSource:0}: Error finding container aa49fca7ff6ceef8b9e14b8827c804d5f1bbbc68773aab1a7eabc4c04fd0e34a: Status 404 returned error can't find the container with id aa49fca7ff6ceef8b9e14b8827c804d5f1bbbc68773aab1a7eabc4c04fd0e34a Jul 10 20:35:57.708207 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.708132 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhdms\"" Jul 10 20:35:57.716147 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.716122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:35:57.845703 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.845674 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-spzdr"] Jul 10 20:35:57.849696 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:35:57.849667 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995dfafb_2e33_4975_80f9_277f48fd8fec.slice/crio-14614fb72f2390ff849d76e3eb3bb815f29b7d077f0ddd5a80da662381dabf88 WatchSource:0}: Error finding container 14614fb72f2390ff849d76e3eb3bb815f29b7d077f0ddd5a80da662381dabf88: Status 404 returned error can't find the container with id 14614fb72f2390ff849d76e3eb3bb815f29b7d077f0ddd5a80da662381dabf88 Jul 10 20:35:57.960218 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.960135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-spzdr" event={"ID":"995dfafb-2e33-4975-80f9-277f48fd8fec","Type":"ContainerStarted","Data":"14614fb72f2390ff849d76e3eb3bb815f29b7d077f0ddd5a80da662381dabf88"} Jul 10 20:35:57.961140 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:57.961118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lx9r" event={"ID":"092b3231-427e-4ba4-99a2-ca436e68be3b","Type":"ContainerStarted","Data":"aa49fca7ff6ceef8b9e14b8827c804d5f1bbbc68773aab1a7eabc4c04fd0e34a"} Jul 10 20:35:58.861364 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:35:58.861318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:35:58.861869 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:58.861510 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:35:58.861869 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:35:58.861589 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:36:30.861569103 +0000 UTC m=+99.745400425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:36:02.976007 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:02.975967 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-spzdr" event={"ID":"995dfafb-2e33-4975-80f9-277f48fd8fec","Type":"ContainerStarted","Data":"0f3b83c2170d72a78798fe6df3f90e82437169d55941c334935ec2f1d1207995"} Jul 10 20:36:02.976473 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:02.976102 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:36:02.977356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:02.977331 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lx9r" event={"ID":"092b3231-427e-4ba4-99a2-ca436e68be3b","Type":"ContainerStarted","Data":"452cbf1634f93f3f96b948b698b5a76161c0a7e42eed6e52dc2c7fd606397ef3"} Jul 10 20:36:02.990371 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:02.990326 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-spzdr" podStartSLOduration=66.525363328 podStartE2EDuration="1m10.990310553s" totalImagesPullingTime="4.464947225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:36:02.989648872 +0000 UTC m=+71.873480215" watchObservedRunningTime="2026-07-10 20:36:02.990310553 +0000 UTC m=+71.874141896" Jul 10 20:36:03.003646 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:03.003592 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4lx9r" podStartSLOduration=66.221196264 podStartE2EDuration="1m11.003577588s" totalImagesPullingTime="4.782381324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:36:03.003047953 +0000 UTC m=+71.886879296" watchObservedRunningTime="2026-07-10 20:36:03.003577588 +0000 UTC m=+71.887408930" Jul 10 20:36:16.603590 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.603471 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8"] Jul 10 20:36:16.606066 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.606049 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.609226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.609179 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 20:36:16.609226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.609220 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 20:36:16.609402 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.609220 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 20:36:16.609402 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.609268 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 20:36:16.609402 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.609220 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hs7mn\"" Jul 10 20:36:16.614499 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.614477 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8"] Jul 10 20:36:16.682862 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.682828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh5s2\" (UniqueName: \"kubernetes.io/projected/9764d793-78a7-4962-a9fb-0027653e85e8-kube-api-access-fh5s2\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.683017 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.682885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9764d793-78a7-4962-a9fb-0027653e85e8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.783733 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.783701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9764d793-78a7-4962-a9fb-0027653e85e8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.783881 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.783763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh5s2\" (UniqueName: \"kubernetes.io/projected/9764d793-78a7-4962-a9fb-0027653e85e8-kube-api-access-fh5s2\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.786196 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.786175 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9764d793-78a7-4962-a9fb-0027653e85e8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.791986 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.791965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh5s2\" (UniqueName: \"kubernetes.io/projected/9764d793-78a7-4962-a9fb-0027653e85e8-kube-api-access-fh5s2\") pod \"managed-serviceaccount-addon-agent-6584488b6b-dmbp8\" (UID: \"9764d793-78a7-4962-a9fb-0027653e85e8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:16.929541 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:16.929448 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" Jul 10 20:36:17.050108 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:17.050078 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8"] Jul 10 20:36:17.054416 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:36:17.054392 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9764d793_78a7_4962_a9fb_0027653e85e8.slice/crio-47801924db9f706a2b287d8a365843f38a7833df32f8d1e6af36fc21319183aa WatchSource:0}: Error finding container 47801924db9f706a2b287d8a365843f38a7833df32f8d1e6af36fc21319183aa: Status 404 returned error can't find the container with id 47801924db9f706a2b287d8a365843f38a7833df32f8d1e6af36fc21319183aa Jul 10 20:36:18.015768 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:18.015726 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" event={"ID":"9764d793-78a7-4962-a9fb-0027653e85e8","Type":"ContainerStarted","Data":"47801924db9f706a2b287d8a365843f38a7833df32f8d1e6af36fc21319183aa"} Jul 10 20:36:20.023060 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:20.023023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" event={"ID":"9764d793-78a7-4962-a9fb-0027653e85e8","Type":"ContainerStarted","Data":"58284bba4bfa0abcdca3b5673aa3b991aa87099f6b69b2a9bcb700709630e7b5"} Jul 10 20:36:20.037278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:20.037219 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" podStartSLOduration=1.628952425 podStartE2EDuration="4.037198078s" totalImagesPullingTime="2.408245653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:36:20.036849909 +0000 UTC m=+88.920681246" watchObservedRunningTime="2026-07-10 20:36:20.037198078 +0000 UTC m=+88.921029422" Jul 10 20:36:30.881312 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:30.881230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:36:30.881641 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:36:30.881367 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:36:30.881641 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:36:30.881440 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:37:34.881422949 +0000 UTC m=+163.765254270 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:36:33.982095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:36:33.982058 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-spzdr" Jul 10 20:37:01.384974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:37:01.384939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:37:01.385453 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:01.385048 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 20:37:01.385453 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:01.385111 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs podName:04669449-ce1e-40ad-804b-7b1bb3f744c6 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:03.385096418 +0000 UTC m=+252.268927739 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs") pod "network-metrics-daemon-w5tk4" (UID: "04669449-ce1e-40ad-804b-7b1bb3f744c6") : secret "metrics-daemon-secret" not found Jul 10 20:37:30.009716 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:30.009673 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xtph7" podUID="2d544536-846f-4bd5-a1af-1d9b4f8bc3a2" Jul 10 20:37:30.190070 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:37:30.190048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtph7" Jul 10 20:37:31.611827 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:31.611776 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w5tk4" podUID="04669449-ce1e-40ad-804b-7b1bb3f744c6" Jul 10 20:37:34.902889 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:37:34.902839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:37:34.903446 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:34.903009 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 20:37:34.903446 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:37:34.903107 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls podName:2d544536-846f-4bd5-a1af-1d9b4f8bc3a2 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:36.903083319 +0000 UTC m=+285.786914640 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls") pod "dns-default-xtph7" (UID: "2d544536-846f-4bd5-a1af-1d9b4f8bc3a2") : secret "dns-default-metrics-tls" not found Jul 10 20:37:44.590992 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:37:44.590910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:38:19.806986 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:19.806954 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:20.313827 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:20.313783 2566 generic.go:356] "Generic (PLEG): container finished" podID="9764d793-78a7-4962-a9fb-0027653e85e8" containerID="58284bba4bfa0abcdca3b5673aa3b991aa87099f6b69b2a9bcb700709630e7b5" exitCode=255 Jul 10 20:38:20.313988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:20.313859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" event={"ID":"9764d793-78a7-4962-a9fb-0027653e85e8","Type":"ContainerDied","Data":"58284bba4bfa0abcdca3b5673aa3b991aa87099f6b69b2a9bcb700709630e7b5"} Jul 10 20:38:20.319292 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:20.319273 2566 scope.go:117] "RemoveContainer" containerID="58284bba4bfa0abcdca3b5673aa3b991aa87099f6b69b2a9bcb700709630e7b5" Jul 10 20:38:21.318871 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:21.318835 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6584488b6b-dmbp8" event={"ID":"9764d793-78a7-4962-a9fb-0027653e85e8","Type":"ContainerStarted","Data":"fcd55cc55be255e49603977573c0de035248b4aa82804cf6e91af0a2a84543a4"} Jul 10 20:38:44.868730 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.868682 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:38:44.870363 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.870349 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.872787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.872758 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 20:38:44.873661 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.873639 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 20:38:44.873742 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.873645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 20:38:44.873742 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.873667 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7k2jt\"" Jul 10 20:38:44.873839 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.873784 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-9f8879f46-24676"] Jul 10 20:38:44.875509 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.875490 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.877683 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.877665 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 20:38:44.878225 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878209 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 20:38:44.878307 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878203 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 20:38:44.878449 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878437 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 20:38:44.878515 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878496 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 20:38:44.878864 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878852 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 20:38:44.878914 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.878884 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-t86gf\"" Jul 10 20:38:44.881404 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.881386 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 20:38:44.892460 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.892435 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:38:44.895592 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.895574 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9f8879f46-24676"] Jul 10 20:38:44.961706 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8wn5\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.961905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.961905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.961905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961842 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.961905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961880 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.961905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961901 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-stats-auth\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv6pl\" (UniqueName: \"kubernetes.io/projected/35d11801-b792-4e8f-b178-2c9d0c3cd49a-kube-api-access-pv6pl\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.961996 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.962087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.962121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.962117 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:44.962304 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.962135 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-default-certificate\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:44.970107 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.970083 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq"] Jul 10 20:38:44.972193 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.972178 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-764xs"] Jul 10 20:38:44.972319 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.972304 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:44.974120 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.974101 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:44.977359 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.977337 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 20:38:44.977359 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.977348 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 20:38:44.978392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978373 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 20:38:44.978534 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978516 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 20:38:44.978611 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978377 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 20:38:44.978611 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978394 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:38:44.978718 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978448 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 20:38:44.978718 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978456 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 20:38:44.978718 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978377 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-g4nbm\"" Jul 10 20:38:44.978995 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.978524 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2lh8f\"" Jul 10 20:38:44.982551 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.982536 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 20:38:44.986868 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.986849 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-764xs"] Jul 10 20:38:44.987755 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:44.987734 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq"] Jul 10 20:38:45.062409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.062409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-default-certificate\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6sl\" (UniqueName: \"kubernetes.io/projected/b64ece5b-3549-48e5-bce7-33827927b4f1-kube-api-access-9g6sl\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv6pl\" (UniqueName: \"kubernetes.io/projected/35d11801-b792-4e8f-b178-2c9d0c3cd49a-kube-api-access-pv6pl\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.062619 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b64ece5b-3549-48e5-bce7-33827927b4f1-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.062662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.062685 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:45.562664196 +0000 UTC m=+234.446495523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : secret "router-metrics-certs-default" not found Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.062787 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.062908 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.062924 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bff4d7977-6smbs: secret "image-registry-tls" not found Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.062989 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls podName:edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:45.562972019 +0000 UTC m=+234.446803343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls") pod "image-registry-bff4d7977-6smbs" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6") : secret "image-registry-tls" not found Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcm87\" (UniqueName: \"kubernetes.io/projected/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-kube-api-access-hcm87\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wn5\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-trusted-ca\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.063243 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063263 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-serving-cert\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-config\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-stats-auth\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.063521 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:45.563504579 +0000 UTC m=+234.447335916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : configmap references non-existent config key: service-ca.crt Jul 10 20:38:45.063702 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.064019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.063718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.065122 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.065102 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.065410 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.065385 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.065511 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.065490 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-default-certificate\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.065658 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.065642 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-stats-auth\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.071226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.071182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.071226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.071183 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wn5\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.071412 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.071394 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv6pl\" (UniqueName: \"kubernetes.io/projected/35d11801-b792-4e8f-b178-2c9d0c3cd49a-kube-api-access-pv6pl\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.103651 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.103620 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:45.164481 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6sl\" (UniqueName: \"kubernetes.io/projected/b64ece5b-3549-48e5-bce7-33827927b4f1-kube-api-access-9g6sl\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.164481 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b64ece5b-3549-48e5-bce7-33827927b4f1-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.164664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcm87\" (UniqueName: \"kubernetes.io/projected/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-kube-api-access-hcm87\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.164664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-trusted-ca\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.164664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.164664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164545 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-serving-cert\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.164664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.164563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-config\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.165021 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.164670 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:45.165021 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.164739 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:45.664719308 +0000 UTC m=+234.548550646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:45.165409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.165388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-config\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.165409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.165402 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b64ece5b-3549-48e5-bce7-33827927b4f1-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.165480 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.165431 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-trusted-ca\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.167050 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.167028 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-serving-cert\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.174510 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.174482 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6sl\" (UniqueName: \"kubernetes.io/projected/b64ece5b-3549-48e5-bce7-33827927b4f1-kube-api-access-9g6sl\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.174884 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.174865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcm87\" (UniqueName: \"kubernetes.io/projected/8afb6140-2e68-4204-8c67-8f0e6c4cc4c3-kube-api-access-hcm87\") pod \"console-operator-7866497494-764xs\" (UID: \"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3\") " pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.205131 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.205104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:45.287457 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.287422 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:45.408867 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.408840 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-764xs"] Jul 10 20:38:45.412111 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:38:45.412085 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afb6140_2e68_4204_8c67_8f0e6c4cc4c3.slice/crio-e0902a0783f0124ef0b71a20dc6fa320f711e0ba5dbd1c3e1765381e0ad8621e WatchSource:0}: Error finding container e0902a0783f0124ef0b71a20dc6fa320f711e0ba5dbd1c3e1765381e0ad8621e: Status 404 returned error can't find the container with id e0902a0783f0124ef0b71a20dc6fa320f711e0ba5dbd1c3e1765381e0ad8621e Jul 10 20:38:45.568144 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.568104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.568321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.568156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:45.568321 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568251 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:38:45.568321 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568258 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:38:45.568460 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.568318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:45.568460 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568334 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:46.568312442 +0000 UTC m=+235.452143781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : secret "router-metrics-certs-default" not found Jul 10 20:38:45.568460 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568262 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bff4d7977-6smbs: secret "image-registry-tls" not found Jul 10 20:38:45.568460 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568401 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:46.568387456 +0000 UTC m=+235.452218787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : configmap references non-existent config key: service-ca.crt Jul 10 20:38:45.568460 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.568415 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls podName:edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:46.568409126 +0000 UTC m=+235.452240447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls") pod "image-registry-bff4d7977-6smbs" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6") : secret "image-registry-tls" not found Jul 10 20:38:45.669724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:45.669633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:45.669897 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.669778 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:45.669897 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:45.669867 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:46.669850517 +0000 UTC m=+235.553681837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:46.388974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:46.388937 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-764xs" event={"ID":"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3","Type":"ContainerStarted","Data":"e0902a0783f0124ef0b71a20dc6fa320f711e0ba5dbd1c3e1765381e0ad8621e"} Jul 10 20:38:46.578610 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:46.578570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:46.578660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:46.578704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578716 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578736 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bff4d7977-6smbs: secret "image-registry-tls" not found Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578785 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls podName:edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.578769467 +0000 UTC m=+237.462600792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls") pod "image-registry-bff4d7977-6smbs" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6") : secret "image-registry-tls" not found Jul 10 20:38:46.578827 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578801 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:38:46.579071 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578819 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.578791554 +0000 UTC m=+237.462622876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : configmap references non-existent config key: service-ca.crt Jul 10 20:38:46.579071 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.578894 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.57887715 +0000 UTC m=+237.462708485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : secret "router-metrics-certs-default" not found Jul 10 20:38:46.679844 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:46.679751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:46.679979 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.679898 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:46.679979 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:46.679959 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:48.679942855 +0000 UTC m=+237.563774176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:47.668705 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:47.668622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:47.728767 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:47.728732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:48.396376 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.396347 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/0.log" Jul 10 20:38:48.396545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.396390 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb6140-2e68-4204-8c67-8f0e6c4cc4c3" containerID="6b6340dbd40254dac62867547ea555d74b8ad3d90de1d19586666e190b42d2d8" exitCode=255 Jul 10 20:38:48.396545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.396469 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-764xs" event={"ID":"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3","Type":"ContainerDied","Data":"6b6340dbd40254dac62867547ea555d74b8ad3d90de1d19586666e190b42d2d8"} Jul 10 20:38:48.396748 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.396731 2566 scope.go:117] "RemoveContainer" containerID="6b6340dbd40254dac62867547ea555d74b8ad3d90de1d19586666e190b42d2d8" Jul 10 20:38:48.594707 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.594669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:48.594903 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.594719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:48.594903 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.594826 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:38:48.594903 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.594876 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:52.594861121 +0000 UTC m=+241.478692445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : secret "router-metrics-certs-default" not found Jul 10 20:38:48.594903 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.594895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:48.595077 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.594968 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:38:52.594961533 +0000 UTC m=+241.478792854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : configmap references non-existent config key: service-ca.crt Jul 10 20:38:48.595077 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.594993 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:38:48.595077 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.595016 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bff4d7977-6smbs: secret "image-registry-tls" not found Jul 10 20:38:48.595077 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.595063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls podName:edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:52.595048554 +0000 UTC m=+241.478879878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls") pod "image-registry-bff4d7977-6smbs" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6") : secret "image-registry-tls" not found Jul 10 20:38:48.695531 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:48.695450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:48.695905 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.695598 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:48.695905 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:48.695666 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:52.695647928 +0000 UTC m=+241.579479256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:49.338143 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.338111 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kmrzc"] Jul 10 20:38:49.341018 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.341001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.344149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.344127 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 20:38:49.344264 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.344216 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 20:38:49.344509 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.344486 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n7x97\"" Jul 10 20:38:49.344620 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.344527 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 20:38:49.344768 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.344751 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 20:38:49.357039 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.357017 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmrzc"] Jul 10 20:38:49.376432 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.376404 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:49.399497 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.399472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46l6q\" (UniqueName: \"kubernetes.io/projected/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-api-access-46l6q\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.399624 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.399527 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/278c6a6e-33a5-4918-9976-4dbe8b1223a8-crio-socket\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.399624 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.399580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.399729 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.399623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.399729 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.399694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/278c6a6e-33a5-4918-9976-4dbe8b1223a8-data-volume\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.400750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.400734 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:38:49.401138 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.401124 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/0.log" Jul 10 20:38:49.401201 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.401159 2566 generic.go:356] "Generic (PLEG): container finished" podID="8afb6140-2e68-4204-8c67-8f0e6c4cc4c3" containerID="789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad" exitCode=255 Jul 10 20:38:49.401250 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.401193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-764xs" event={"ID":"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3","Type":"ContainerDied","Data":"789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad"} Jul 10 20:38:49.401302 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.401259 2566 scope.go:117] "RemoveContainer" containerID="6b6340dbd40254dac62867547ea555d74b8ad3d90de1d19586666e190b42d2d8" Jul 10 20:38:49.401450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.401425 2566 scope.go:117] "RemoveContainer" containerID="789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad" Jul 10 20:38:49.401614 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:49.401593 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-764xs_openshift-console-operator(8afb6140-2e68-4204-8c67-8f0e6c4cc4c3)\"" pod="openshift-console-operator/console-operator-7866497494-764xs" podUID="8afb6140-2e68-4204-8c67-8f0e6c4cc4c3" Jul 10 20:38:49.434460 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.434437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:49.500863 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.500827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/278c6a6e-33a5-4918-9976-4dbe8b1223a8-data-volume\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501000 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.500895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46l6q\" (UniqueName: \"kubernetes.io/projected/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-api-access-46l6q\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501000 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.500959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/278c6a6e-33a5-4918-9976-4dbe8b1223a8-crio-socket\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.501047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.501071 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/278c6a6e-33a5-4918-9976-4dbe8b1223a8-crio-socket\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.501091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501272 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:49.501143 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:38:49.501272 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:49.501168 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:38:49.501272 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.501173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/278c6a6e-33a5-4918-9976-4dbe8b1223a8-data-volume\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.501272 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:49.501224 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.001203453 +0000 UTC m=+238.885034779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : configmap "kube-rbac-proxy" not found Jul 10 20:38:49.501272 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:49.501241 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:50.001233203 +0000 UTC m=+238.885064525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : secret "insights-runtime-extractor-tls" not found Jul 10 20:38:49.509288 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.509261 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46l6q\" (UniqueName: \"kubernetes.io/projected/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-api-access-46l6q\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:49.678428 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.678323 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:49.779345 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:49.779305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.005090 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.005046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:50.005090 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.005103 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:50.005298 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:50.005170 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:38:50.005298 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:50.005219 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:38:50.005298 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:50.005237 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:51.005220056 +0000 UTC m=+239.889051377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : configmap "kube-rbac-proxy" not found Jul 10 20:38:50.005298 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:50.005260 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:51.005247711 +0000 UTC m=+239.889079033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : secret "insights-runtime-extractor-tls" not found Jul 10 20:38:50.118254 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.118221 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.175646 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.175616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.386786 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.386713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.405259 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.405236 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:38:50.405554 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.405541 2566 scope.go:117] "RemoveContainer" containerID="789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad" Jul 10 20:38:50.405696 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:50.405679 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-764xs_openshift-console-operator(8afb6140-2e68-4204-8c67-8f0e6c4cc4c3)\"" pod="openshift-console-operator/console-operator-7866497494-764xs" podUID="8afb6140-2e68-4204-8c67-8f0e6c4cc4c3" Jul 10 20:38:50.443347 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.443321 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.612642 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.612608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.711100 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.711020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:50.812155 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:50.812122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:51.012989 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:51.012958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:51.013152 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:51.013003 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:51.013152 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:51.013074 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:38:51.013152 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:51.013138 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:53.013123453 +0000 UTC m=+241.896954778 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : configmap "kube-rbac-proxy" not found Jul 10 20:38:51.013152 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:51.013138 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:38:51.013309 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:51.013164 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:53.013158685 +0000 UTC m=+241.896990006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : secret "insights-runtime-extractor-tls" not found Jul 10 20:38:51.107241 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:51.107217 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hfcmm_a341f91e-55e6-42a5-b430-2b2c3b2827e9/dns-node-resolver/0.log" Jul 10 20:38:52.015917 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.015885 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:52.084896 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.084865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:52.508523 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.508500 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wlsl8_1d1408fb-0169-487a-ac62-3cdf58ef7c1d/node-ca/0.log" Jul 10 20:38:52.624024 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.623993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:52.624175 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.624038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:38:52.624175 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.624092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:38:52.624175 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624129 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 20:38:52.624270 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624190 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 20:38:52.624270 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624203 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bff4d7977-6smbs: secret "image-registry-tls" not found Jul 10 20:38:52.624270 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624191 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:39:00.624175651 +0000 UTC m=+249.508006973 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : secret "router-metrics-certs-default" not found Jul 10 20:38:52.624270 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624251 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle podName:35d11801-b792-4e8f-b178-2c9d0c3cd49a nodeName:}" failed. No retries permitted until 2026-07-10 20:39:00.624234777 +0000 UTC m=+249.508066097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle") pod "router-default-9f8879f46-24676" (UID: "35d11801-b792-4e8f-b178-2c9d0c3cd49a") : configmap references non-existent config key: service-ca.crt Jul 10 20:38:52.624400 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.624272 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls podName:edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:00.624262213 +0000 UTC m=+249.508093540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls") pod "image-registry-bff4d7977-6smbs" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6") : secret "image-registry-tls" not found Jul 10 20:38:52.704319 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.704290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:52.724485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.724459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:38:52.724573 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.724560 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:52.724619 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:52.724610 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:00.724597189 +0000 UTC m=+249.608428509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:38:52.804838 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.804741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:52.880995 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.880962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:52.906911 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:52.906891 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2tjkp_58d1a662-8361-4825-b188-9d85c7f5ef94/serve-healthcheck-canary/0.log" Jul 10 20:38:53.006975 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:53.006949 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:53.026030 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:53.026004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:53.026336 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:53.026043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:53.026336 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:53.026133 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:38:53.026336 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:53.026175 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:38:53.026336 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:53.026189 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:57.02617421 +0000 UTC m=+245.910005531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : configmap "kube-rbac-proxy" not found Jul 10 20:38:53.026336 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:53.026213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:57.026202376 +0000 UTC m=+245.910033702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : secret "insights-runtime-extractor-tls" not found Jul 10 20:38:53.110043 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:53.109975 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:54.053700 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:54.053667 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:54.153821 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:54.153789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:54.255568 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:54.255540 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:54.360897 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:54.360799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:54.428544 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:54.428513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:55.144565 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.144533 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:55.288371 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.288335 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:55.288562 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.288421 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:38:55.288851 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.288801 2566 scope.go:117] "RemoveContainer" containerID="789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad" Jul 10 20:38:55.289023 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:55.289005 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-764xs_openshift-console-operator(8afb6140-2e68-4204-8c67-8f0e6c4cc4c3)\"" pod="openshift-console-operator/console-operator-7866497494-764xs" podUID="8afb6140-2e68-4204-8c67-8f0e6c4cc4c3" Jul 10 20:38:55.545125 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.545099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:55.652799 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.652771 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:55.791896 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:55.791867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:56.165112 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:56.165077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:56.745594 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:56.745566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:57.057988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.057898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:57.057988 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.057945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:38:57.058183 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:57.058023 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 20:38:57.058183 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:57.058047 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 20:38:57.058183 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:57.058089 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:05.058070729 +0000 UTC m=+253.941902056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : configmap "kube-rbac-proxy" not found Jul 10 20:38:57.058183 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:38:57.058104 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls podName:278c6a6e-33a5-4918-9976-4dbe8b1223a8 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:05.058097921 +0000 UTC m=+253.941929242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmrzc" (UID: "278c6a6e-33a5-4918-9976-4dbe8b1223a8") : secret "insights-runtime-extractor-tls" not found Jul 10 20:38:57.350964 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.350878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:57.542844 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.542797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:57.755905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.755876 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:57.942847 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:57.942817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:58.149975 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:58.149892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:58.249849 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:58.249804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:58.751998 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:58.751968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:59.347932 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:59.347904 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:59.547930 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:59.547898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:38:59.956226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:38:59.956186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:00.145199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.145155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:00.545543 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.545512 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:00.594707 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.594679 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:00.691355 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.691323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:00.691544 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.691389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:00.691544 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.691437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:00.692134 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.692114 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35d11801-b792-4e8f-b178-2c9d0c3cd49a-service-ca-bundle\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:00.693822 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.693785 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"image-registry-bff4d7977-6smbs\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:00.693962 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.693943 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d11801-b792-4e8f-b178-2c9d0c3cd49a-metrics-certs\") pod \"router-default-9f8879f46-24676\" (UID: \"35d11801-b792-4e8f-b178-2c9d0c3cd49a\") " pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:00.782442 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.782406 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7k2jt\"" Jul 10 20:39:00.787900 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.787880 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-t86gf\"" Jul 10 20:39:00.790709 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.790694 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:00.792121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.792102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:39:00.792287 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:00.792269 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 20:39:00.792354 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:00.792345 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls podName:b64ece5b-3549-48e5-bce7-33827927b4f1 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:16.792322605 +0000 UTC m=+265.676153932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-kb7cq" (UID: "b64ece5b-3549-48e5-bce7-33827927b4f1") : secret "cluster-monitoring-operator-tls" not found Jul 10 20:39:00.796901 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.796846 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:00.928694 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.928667 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:39:00.931904 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:00.931877 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd6fe05_04fc_404a_86c1_6c0a2d6fc0b6.slice/crio-67b3fe842e6b6432bc50a770ebcda967ad11be997b9342fc1767a9694a81828b WatchSource:0}: Error finding container 67b3fe842e6b6432bc50a770ebcda967ad11be997b9342fc1767a9694a81828b: Status 404 returned error can't find the container with id 67b3fe842e6b6432bc50a770ebcda967ad11be997b9342fc1767a9694a81828b Jul 10 20:39:00.940780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:00.940723 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-9f8879f46-24676"] Jul 10 20:39:00.943206 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:00.943186 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d11801_b792_4e8f_b178_2c9d0c3cd49a.slice/crio-2cf4ba5c7a005e47e042725e92ec5057e70f84bb3d250c0eafa2c0f72b6e21a9 WatchSource:0}: Error finding container 2cf4ba5c7a005e47e042725e92ec5057e70f84bb3d250c0eafa2c0f72b6e21a9: Status 404 returned error can't find the container with id 2cf4ba5c7a005e47e042725e92ec5057e70f84bb3d250c0eafa2c0f72b6e21a9 Jul 10 20:39:01.149122 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.149043 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:01.436518 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.436428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9f8879f46-24676" event={"ID":"35d11801-b792-4e8f-b178-2c9d0c3cd49a","Type":"ContainerStarted","Data":"99e7da8728882bb2b7634a03be611289418c9d9191c65cc2a1da820d78aece9a"} Jul 10 20:39:01.436518 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.436484 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-9f8879f46-24676" event={"ID":"35d11801-b792-4e8f-b178-2c9d0c3cd49a","Type":"ContainerStarted","Data":"2cf4ba5c7a005e47e042725e92ec5057e70f84bb3d250c0eafa2c0f72b6e21a9"} Jul 10 20:39:01.437802 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.437779 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" event={"ID":"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6","Type":"ContainerStarted","Data":"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177"} Jul 10 20:39:01.437928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.437840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" event={"ID":"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6","Type":"ContainerStarted","Data":"67b3fe842e6b6432bc50a770ebcda967ad11be997b9342fc1767a9694a81828b"} Jul 10 20:39:01.437928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.437861 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:01.459003 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.458963 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-9f8879f46-24676" podStartSLOduration=17.458947452 podStartE2EDuration="17.458947452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:01.458007441 +0000 UTC m=+250.341838783" watchObservedRunningTime="2026-07-10 20:39:01.458947452 +0000 UTC m=+250.342778773" Jul 10 20:39:01.476928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.476884 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" podStartSLOduration=17.476869903 podStartE2EDuration="17.476869903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:01.475780556 +0000 UTC m=+250.359611910" watchObservedRunningTime="2026-07-10 20:39:01.476869903 +0000 UTC m=+250.360701246" Jul 10 20:39:01.547188 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.547150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:01.746076 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.746041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:01.797115 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.797078 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:01.799684 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:01.799659 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:02.184181 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:02.184096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:02.441149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:02.441065 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:02.442213 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:02.442197 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-9f8879f46-24676" Jul 10 20:39:02.538886 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:02.538852 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:02.589428 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:02.589387 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:03.145475 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.145444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:03.411925 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.411839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:39:03.414105 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.414080 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04669449-ce1e-40ad-804b-7b1bb3f744c6-metrics-certs\") pod \"network-metrics-daemon-w5tk4\" (UID: \"04669449-ce1e-40ad-804b-7b1bb3f744c6\") " pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:39:03.493802 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.493773 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rvkjn\"" Jul 10 20:39:03.501683 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.501659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w5tk4" Jul 10 20:39:03.605491 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.605464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:03.625542 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.625516 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w5tk4"] Jul 10 20:39:03.628937 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:03.628913 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04669449_ce1e_40ad_804b_7b1bb3f744c6.slice/crio-7ad3035d015a314bb12729cf59fc2d39bbf1c459f32d93e3ee90d0dec15823bd WatchSource:0}: Error finding container 7ad3035d015a314bb12729cf59fc2d39bbf1c459f32d93e3ee90d0dec15823bd: Status 404 returned error can't find the container with id 7ad3035d015a314bb12729cf59fc2d39bbf1c459f32d93e3ee90d0dec15823bd Jul 10 20:39:03.949053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:03.949024 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:04.088504 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:04.088473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:04.153314 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:04.153278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:04.449239 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:04.449206 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5tk4" event={"ID":"04669449-ce1e-40ad-804b-7b1bb3f744c6","Type":"ContainerStarted","Data":"7ad3035d015a314bb12729cf59fc2d39bbf1c459f32d93e3ee90d0dec15823bd"} Jul 10 20:39:04.547609 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:04.547566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:04.745600 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:04.745553 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:05.127540 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.127506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:39:05.127540 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.127541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:39:05.128049 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.128029 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/278c6a6e-33a5-4918-9976-4dbe8b1223a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:39:05.129761 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.129735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/278c6a6e-33a5-4918-9976-4dbe8b1223a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmrzc\" (UID: \"278c6a6e-33a5-4918-9976-4dbe8b1223a8\") " pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:39:05.252100 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.252070 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n7x97\"" Jul 10 20:39:05.260933 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.260912 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmrzc" Jul 10 20:39:05.382127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.382041 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmrzc"] Jul 10 20:39:05.385705 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:05.385679 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod278c6a6e_33a5_4918_9976_4dbe8b1223a8.slice/crio-2980b0b61b6b4335a125635e6cffbaca4360495842b25968bb9114fdeea41afb WatchSource:0}: Error finding container 2980b0b61b6b4335a125635e6cffbaca4360495842b25968bb9114fdeea41afb: Status 404 returned error can't find the container with id 2980b0b61b6b4335a125635e6cffbaca4360495842b25968bb9114fdeea41afb Jul 10 20:39:05.453467 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.453432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5tk4" event={"ID":"04669449-ce1e-40ad-804b-7b1bb3f744c6","Type":"ContainerStarted","Data":"14d0fa3be1b84ce8eff40c35db394578771f06e5e04e48e969ce65e86cc27dd4"} Jul 10 20:39:05.453916 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.453491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w5tk4" event={"ID":"04669449-ce1e-40ad-804b-7b1bb3f744c6","Type":"ContainerStarted","Data":"21ecc3b23bbadcd8e63239b6651a2e7790cfa679ffc0a996da0efd12b29c6fce"} Jul 10 20:39:05.454622 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.454597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmrzc" event={"ID":"278c6a6e-33a5-4918-9976-4dbe8b1223a8","Type":"ContainerStarted","Data":"2980b0b61b6b4335a125635e6cffbaca4360495842b25968bb9114fdeea41afb"} Jul 10 20:39:05.467512 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:05.467462 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w5tk4" podStartSLOduration=252.487940411 podStartE2EDuration="4m13.467445964s" totalImagesPullingTime="979.505553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:05.467243073 +0000 UTC m=+254.351074417" watchObservedRunningTime="2026-07-10 20:39:05.467445964 +0000 UTC m=+254.351277306" Jul 10 20:39:08.590931 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:08.590904 2566 scope.go:117] "RemoveContainer" containerID="789c36768119b3eec98678fb63ac750e4359dc46b2f64a8bc73ae8d67d4a72ad" Jul 10 20:39:09.471532 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.471501 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:39:09.471718 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.471577 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-764xs" event={"ID":"8afb6140-2e68-4204-8c67-8f0e6c4cc4c3","Type":"ContainerStarted","Data":"21a85ea26e7b34ab633de33a7fd990fa19c2863a883ccab925d6648e1382f18e"} Jul 10 20:39:09.471900 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.471880 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:39:09.473210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.473182 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmrzc" event={"ID":"278c6a6e-33a5-4918-9976-4dbe8b1223a8","Type":"ContainerStarted","Data":"c56d289bbbee384682ca9b091ab0c8d7013cb7e1ee170ffb1182d7f564e56dbb"} Jul 10 20:39:09.487110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.487062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-764xs" podStartSLOduration=23.54693983 podStartE2EDuration="25.487049245s" totalImagesPullingTime="1.940109415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:09.486276525 +0000 UTC m=+258.370107868" watchObservedRunningTime="2026-07-10 20:39:09.487049245 +0000 UTC m=+258.370880588" Jul 10 20:39:09.647674 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.647603 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-764xs" Jul 10 20:39:09.824955 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.824927 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:09.927610 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:09.927514 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.026157 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.026129 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.126325 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.126292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.478471 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.478428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmrzc" event={"ID":"278c6a6e-33a5-4918-9976-4dbe8b1223a8","Type":"ContainerStarted","Data":"68b837091fb04ab725d22400b426a6097cfee655dca08567ac46a1af3750060e"} Jul 10 20:39:10.503727 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.503689 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.556498 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.556461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.925585 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.925552 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:10.983514 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:10.983480 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.314548 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.314512 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.364196 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.364167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.453330 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.453289 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.552117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.552082 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.621107 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.621019 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.716934 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.716902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.753836 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.753789 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:39:11.784420 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.784388 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b7fd9d66-gsjvm"] Jul 10 20:39:11.792930 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.792905 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.800690 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.800628 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b7fd9d66-gsjvm"] Jul 10 20:39:11.808233 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.808208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.880098 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.880023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.978254 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.978219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:11.984073 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984044 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0865e613-a5bf-4383-b8cc-18ff949bdea4-ca-trust-extracted\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984084 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-installation-pull-secrets\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984115 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-tls\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-certificates\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-bound-sa-token\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-image-registry-private-configuration\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t72cb\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-kube-api-access-t72cb\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:11.984485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:11.984319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-trusted-ca\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.085749 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.085711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-image-registry-private-configuration\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.085957 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.085763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t72cb\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-kube-api-access-t72cb\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.085957 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.085867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-trusted-ca\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.085962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0865e613-a5bf-4383-b8cc-18ff949bdea4-ca-trust-extracted\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.085993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-installation-pull-secrets\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.086018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-tls\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.086035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-certificates\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.086051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-bound-sa-token\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.086508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.086480 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0865e613-a5bf-4383-b8cc-18ff949bdea4-ca-trust-extracted\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.087055 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.087014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-trusted-ca\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.087211 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.087116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-certificates\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.089106 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.089082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-registry-tls\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.089217 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.089085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-image-registry-private-configuration\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.089279 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.089218 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0865e613-a5bf-4383-b8cc-18ff949bdea4-installation-pull-secrets\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.094994 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.094968 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-bound-sa-token\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.095215 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.095197 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t72cb\" (UniqueName: \"kubernetes.io/projected/0865e613-a5bf-4383-b8cc-18ff949bdea4-kube-api-access-t72cb\") pod \"image-registry-74b7fd9d66-gsjvm\" (UID: \"0865e613-a5bf-4383-b8cc-18ff949bdea4\") " pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.104039 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.104020 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:12.565761 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:12.565735 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b7fd9d66-gsjvm"] Jul 10 20:39:12.568073 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:12.568042 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0865e613_a5bf_4383_b8cc_18ff949bdea4.slice/crio-1f3982c89e6ebfb1f5d43b09e0dd009b010bfdbe9358ee22e8cdb75314528697 WatchSource:0}: Error finding container 1f3982c89e6ebfb1f5d43b09e0dd009b010bfdbe9358ee22e8cdb75314528697: Status 404 returned error can't find the container with id 1f3982c89e6ebfb1f5d43b09e0dd009b010bfdbe9358ee22e8cdb75314528697 Jul 10 20:39:13.135156 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.135078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:13.195394 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.195354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:13.490630 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.490596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" event={"ID":"0865e613-a5bf-4383-b8cc-18ff949bdea4","Type":"ContainerStarted","Data":"e777051ab6e138266a52c7aeee25cabdaa9206bfc87b8016e0b8e8eb33394c3e"} Jul 10 20:39:13.490803 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.490660 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:13.490803 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.490680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" event={"ID":"0865e613-a5bf-4383-b8cc-18ff949bdea4","Type":"ContainerStarted","Data":"1f3982c89e6ebfb1f5d43b09e0dd009b010bfdbe9358ee22e8cdb75314528697"} Jul 10 20:39:13.492557 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.492529 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmrzc" event={"ID":"278c6a6e-33a5-4918-9976-4dbe8b1223a8","Type":"ContainerStarted","Data":"72d5987c2eebef4e6abc486aa8be5da64e3a656fc6133820ec8eeff47add8f55"} Jul 10 20:39:13.508896 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.508839 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" podStartSLOduration=2.508803145 podStartE2EDuration="2.508803145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:13.507487748 +0000 UTC m=+262.391319090" watchObservedRunningTime="2026-07-10 20:39:13.508803145 +0000 UTC m=+262.392634487" Jul 10 20:39:13.522880 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:13.522833 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kmrzc" podStartSLOduration=17.557545623 podStartE2EDuration="24.522797469s" totalImagesPullingTime="6.965251846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:38:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:13.522278899 +0000 UTC m=+262.406110244" watchObservedRunningTime="2026-07-10 20:39:13.522797469 +0000 UTC m=+262.406628811" Jul 10 20:39:16.823787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:16.823739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:39:16.826292 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:16.826268 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b64ece5b-3549-48e5-bce7-33827927b4f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-kb7cq\" (UID: \"b64ece5b-3549-48e5-bce7-33827927b4f1\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:39:17.084845 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:17.084755 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2lh8f\"" Jul 10 20:39:17.092600 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:17.092577 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" Jul 10 20:39:17.220180 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:17.220123 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq"] Jul 10 20:39:17.222617 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:17.222591 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64ece5b_3549_48e5_bce7_33827927b4f1.slice/crio-64e2076695313206c12b5e514ca55a5f5782e0a542121974253f56b3d8e47cfa WatchSource:0}: Error finding container 64e2076695313206c12b5e514ca55a5f5782e0a542121974253f56b3d8e47cfa: Status 404 returned error can't find the container with id 64e2076695313206c12b5e514ca55a5f5782e0a542121974253f56b3d8e47cfa Jul 10 20:39:17.506665 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:17.506635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" event={"ID":"b64ece5b-3549-48e5-bce7-33827927b4f1","Type":"ContainerStarted","Data":"64e2076695313206c12b5e514ca55a5f5782e0a542121974253f56b3d8e47cfa"} Jul 10 20:39:19.515110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:19.515079 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" event={"ID":"b64ece5b-3549-48e5-bce7-33827927b4f1","Type":"ContainerStarted","Data":"3d49ddf50f4a7b3ce2ad97213a079f63c2cd787790e341d61fcc11862b857961"} Jul 10 20:39:19.518833 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:19.518793 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:19.621118 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:19.621083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:19.694437 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:19.694406 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:19.856473 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:19.856368 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:21.738483 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:21.738447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:21.758804 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:21.758767 2566 patch_prober.go:28] interesting pod/image-registry-bff4d7977-6smbs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 20:39:21.759042 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:21.758860 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:39:21.838485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:21.838456 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:22.849236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:22.849207 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:22.900779 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:22.900746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:22.953267 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:22.953238 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:24.698962 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:24.698925 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:24.766491 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:24.766461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:25.267477 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:25.267449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:25.529925 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:25.529854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:25.578095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:25.578068 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:29.367392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:29.367361 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:29.425499 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:29.425466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:31.758712 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:31.758680 2566 patch_prober.go:28] interesting pod/image-registry-bff4d7977-6smbs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 20:39:31.759103 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:31.758728 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:39:32.108068 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:32.107988 2566 patch_prober.go:28] interesting pod/image-registry-74b7fd9d66-gsjvm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 20:39:32.108068 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:32.108049 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" podUID="0865e613-a5bf-4383-b8cc-18ff949bdea4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 20:39:32.108326 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:32.108302 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:32.209082 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:32.209052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:32.309798 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:32.309764 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:33.191541 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:33.191498 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xtph7" podUID="2d544536-846f-4bd5-a1af-1d9b4f8bc3a2" Jul 10 20:39:33.561053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:33.561023 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:33.724181 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:33.724149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:33.817964 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:33.817880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:33.874282 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:33.874250 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:34.500825 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:34.500783 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b7fd9d66-gsjvm" Jul 10 20:39:34.518538 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:34.518485 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-kb7cq" podStartSLOduration=48.527388424 podStartE2EDuration="50.518466767s" totalImagesPullingTime="1.991078343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:19.530599976 +0000 UTC m=+268.414431322" watchObservedRunningTime="2026-07-10 20:39:34.518466767 +0000 UTC m=+283.402298112" Jul 10 20:39:34.627188 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:34.627157 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:34.726926 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:34.726893 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:34.828956 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:34.828881 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:35.834270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:35.834226 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:36.057537 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.057486 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:36.063824 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.063786 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5"] Jul 10 20:39:36.068233 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.068217 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.070845 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.070801 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 20:39:36.071565 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.071547 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 20:39:36.071617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.071586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qs4np\"" Jul 10 20:39:36.071617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.071576 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 20:39:36.080887 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.080856 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5"] Jul 10 20:39:36.093087 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.093026 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-q4v9b"] Jul 10 20:39:36.096019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.096001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.098382 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.098363 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 20:39:36.098476 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.098402 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 20:39:36.099752 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.099731 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-n87vs\"" Jul 10 20:39:36.099882 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.099871 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 20:39:36.100412 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.100395 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cv28x"] Jul 10 20:39:36.103348 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.103331 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.105374 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.105361 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 20:39:36.106021 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.106000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 20:39:36.106102 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.106042 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 20:39:36.106150 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.106129 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-99dbm\"" Jul 10 20:39:36.111494 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.111470 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-q4v9b"] Jul 10 20:39:36.161098 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-metrics-client-ca\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.161266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-textfile\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-root\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e950d224-6ecb-4395-8124-5df44622e213-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.161396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-sys\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.161396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.161396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnkkf\" (UniqueName: \"kubernetes.io/projected/0a15588c-6e7b-4456-9bf3-d11e304242f9-kube-api-access-pnkkf\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7xzx\" (UniqueName: \"kubernetes.io/projected/e950d224-6ecb-4395-8124-5df44622e213-kube-api-access-m7xzx\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-wtmp\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a2ef422d-131c-4153-8fb2-7eabb17baa8a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqbb\" (UniqueName: \"kubernetes.io/projected/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-api-access-bzqbb\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.161634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.161622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.261951 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.261915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.261951 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.261950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqbb\" (UniqueName: \"kubernetes.io/projected/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-api-access-bzqbb\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262202 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.261969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262202 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262202 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-metrics-client-ca\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262202 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262202 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-textfile\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-root\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262265 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e950d224-6ecb-4395-8124-5df44622e213-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-sys\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.262403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnkkf\" (UniqueName: \"kubernetes.io/projected/0a15588c-6e7b-4456-9bf3-d11e304242f9-kube-api-access-pnkkf\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7xzx\" (UniqueName: \"kubernetes.io/projected/e950d224-6ecb-4395-8124-5df44622e213-kube-api-access-m7xzx\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-wtmp\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262538 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-textfile\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a2ef422d-131c-4153-8fb2-7eabb17baa8a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262616 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-accelerators-collector-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.262647 2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 10 20:39:36.262750 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.262713 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls podName:a2ef422d-131c-4153-8fb2-7eabb17baa8a nodeName:}" failed. No retries permitted until 2026-07-10 20:39:36.76269484 +0000 UTC m=+285.646526165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-q4v9b" (UID: "a2ef422d-131c-4153-8fb2-7eabb17baa8a") : secret "kube-state-metrics-tls" not found Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.262779 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.262844 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls podName:0a15588c-6e7b-4456-9bf3-d11e304242f9 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:36.76280209 +0000 UTC m=+285.646633414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls") pod "node-exporter-cv28x" (UID: "0a15588c-6e7b-4456-9bf3-d11e304242f9") : secret "node-exporter-tls" not found Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a15588c-6e7b-4456-9bf3-d11e304242f9-metrics-client-ca\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262909 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a2ef422d-131c-4153-8fb2-7eabb17baa8a-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262904 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.262945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-root\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.263004 2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.263045 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls podName:e950d224-6ecb-4395-8124-5df44622e213 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:36.763030404 +0000 UTC m=+285.646861726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-bqnp5" (UID: "e950d224-6ecb-4395-8124-5df44622e213") : secret "openshift-state-metrics-tls" not found Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.263039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-wtmp\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.263278 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.263098 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a15588c-6e7b-4456-9bf3-d11e304242f9-sys\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.263772 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.263461 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef422d-131c-4153-8fb2-7eabb17baa8a-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.263772 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.263515 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e950d224-6ecb-4395-8124-5df44622e213-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.264677 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.264656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.264950 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.264930 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.265155 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.265134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.271740 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.271718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqbb\" (UniqueName: \"kubernetes.io/projected/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-api-access-bzqbb\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.271878 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.271801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnkkf\" (UniqueName: \"kubernetes.io/projected/0a15588c-6e7b-4456-9bf3-d11e304242f9-kube-api-access-pnkkf\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.271936 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.271913 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7xzx\" (UniqueName: \"kubernetes.io/projected/e950d224-6ecb-4395-8124-5df44622e213-kube-api-access-m7xzx\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.766946 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.766915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:36.767131 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.766972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.767131 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.767003 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.767131 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.767057 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 20:39:36.767131 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:36.767129 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls podName:0a15588c-6e7b-4456-9bf3-d11e304242f9 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:37.767113616 +0000 UTC m=+286.650944938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls") pod "node-exporter-cv28x" (UID: "0a15588c-6e7b-4456-9bf3-d11e304242f9") : secret "node-exporter-tls" not found Jul 10 20:39:36.769422 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.769388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2ef422d-131c-4153-8fb2-7eabb17baa8a-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-q4v9b\" (UID: \"a2ef422d-131c-4153-8fb2-7eabb17baa8a\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:36.769551 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.769456 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e950d224-6ecb-4395-8124-5df44622e213-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-bqnp5\" (UID: \"e950d224-6ecb-4395-8124-5df44622e213\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:36.777893 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.777838 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" containerID="cri-o://b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177" gracePeriod=30 Jul 10 20:39:36.968473 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.968444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:36.970659 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.970641 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d544536-846f-4bd5-a1af-1d9b4f8bc3a2-metrics-tls\") pod \"dns-default-xtph7\" (UID: \"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2\") " pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:36.977685 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:36.977663 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" Jul 10 20:39:37.005078 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.005044 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" Jul 10 20:39:37.012451 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.012424 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:37.069233 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069198 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069416 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069240 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069416 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069266 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069416 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069327 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069416 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069372 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069416 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069407 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069766 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069437 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8wn5\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.069766 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.069468 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration\") pod \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\" (UID: \"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6\") " Jul 10 20:39:37.070652 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.070357 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:39:37.070652 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.070382 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:39:37.078720 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.078649 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:39:37.078961 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.078933 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5" (OuterVolumeSpecName: "kube-api-access-g8wn5") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "kube-api-access-g8wn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:39:37.079192 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.079168 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:39:37.083383 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.083343 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:39:37.084854 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.084832 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:39:37.089161 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.089133 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" (UID: "edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:39:37.132433 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.132342 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5"] Jul 10 20:39:37.141380 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.141356 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 20:39:37.141657 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.141641 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" Jul 10 20:39:37.141736 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.141660 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" Jul 10 20:39:37.141736 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.141729 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerName="registry" Jul 10 20:39:37.146907 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.145988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.148306 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.148283 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-q4v9b"] Jul 10 20:39:37.148743 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.148728 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 20:39:37.149168 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149148 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 20:39:37.149263 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149172 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 20:39:37.149381 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149363 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 20:39:37.149443 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149366 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 20:39:37.149538 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149524 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 20:39:37.149676 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149665 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lwr47\"" Jul 10 20:39:37.149727 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149708 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 20:39:37.149985 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149829 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 20:39:37.149985 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.149876 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 20:39:37.152060 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:37.152000 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ef422d_131c_4153_8fb2_7eabb17baa8a.slice/crio-15c6aec9dfcc30d4dc08f583356bfe97d9ba1201bb377e67f15f424f5b8314f8 WatchSource:0}: Error finding container 15c6aec9dfcc30d4dc08f583356bfe97d9ba1201bb377e67f15f424f5b8314f8: Status 404 returned error can't find the container with id 15c6aec9dfcc30d4dc08f583356bfe97d9ba1201bb377e67f15f424f5b8314f8 Jul 10 20:39:37.161622 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.161598 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 20:39:37.163469 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.163450 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j2m8z\"" Jul 10 20:39:37.170795 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.170920 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170825 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.170920 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.170920 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.170982 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171209 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-web-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171209 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171125 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn2br\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-kube-api-access-bn2br\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171209 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171185 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-config-volume\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171346 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171346 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171346 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-config-out\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171367 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-certificates\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171390 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-bound-sa-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171405 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-trusted-ca\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171418 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8wn5\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-kube-api-access-g8wn5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171432 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-image-registry-private-configuration\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171450 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171446 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-ca-trust-extracted\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171459 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-registry-tls\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171474 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6-installation-pull-secrets\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:39:37.171719 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.171711 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-web-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn2br\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-kube-api-access-bn2br\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-config-volume\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-config-out\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.272982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:37.272864 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:37.273387 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls podName:67cd844e-96f3-4352-8975-d84f72dd484c nodeName:}" failed. No retries permitted until 2026-07-10 20:39:37.773354475 +0000 UTC m=+286.657185812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "67cd844e-96f3-4352-8975-d84f72dd484c") : secret "alertmanager-main-tls" not found Jul 10 20:39:37.274508 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.273951 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.275477 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:37.275015 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle podName:67cd844e-96f3-4352-8975-d84f72dd484c nodeName:}" failed. No retries permitted until 2026-07-10 20:39:37.77497902 +0000 UTC m=+286.658810344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "67cd844e-96f3-4352-8975-d84f72dd484c") : configmap references non-existent config key: ca-bundle.crt Jul 10 20:39:37.275956 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.275897 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.279216 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.279130 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.280097 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.279951 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.280678 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.280657 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.282095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.281413 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.282095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.281565 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.282095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.281920 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-config-volume\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.282095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.282011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-web-config\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.283677 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.283636 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn2br\" (UniqueName: \"kubernetes.io/projected/67cd844e-96f3-4352-8975-d84f72dd484c-kube-api-access-bn2br\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.284594 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.284572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67cd844e-96f3-4352-8975-d84f72dd484c-config-out\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.327899 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.327866 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xtph7"] Jul 10 20:39:37.330914 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:37.330877 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d544536_846f_4bd5_a1af_1d9b4f8bc3a2.slice/crio-936be7f0ec082aa62d4119436712cf3193153e13d302ee80b8cf636748948023 WatchSource:0}: Error finding container 936be7f0ec082aa62d4119436712cf3193153e13d302ee80b8cf636748948023: Status 404 returned error can't find the container with id 936be7f0ec082aa62d4119436712cf3193153e13d302ee80b8cf636748948023 Jul 10 20:39:37.337400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.337377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:37.437929 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.437894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:37.576884 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.576776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" event={"ID":"e950d224-6ecb-4395-8124-5df44622e213","Type":"ContainerStarted","Data":"981f27e10b5adf9548293ad63c9084378bf83cf9f44a2a64a4b76fe4371313d5"} Jul 10 20:39:37.576884 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.576866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" event={"ID":"e950d224-6ecb-4395-8124-5df44622e213","Type":"ContainerStarted","Data":"0df0e661fec42797261a8c9622acf2f695c8c39be38b73069851508e6b19eb27"} Jul 10 20:39:37.577084 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.576897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" event={"ID":"e950d224-6ecb-4395-8124-5df44622e213","Type":"ContainerStarted","Data":"8bf3ba90f01cfc95020446771ef65b9e065ffc691913f160df81973c63e407f5"} Jul 10 20:39:37.577857 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.577833 2566 generic.go:356] "Generic (PLEG): container finished" podID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" containerID="b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177" exitCode=0 Jul 10 20:39:37.577963 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.577925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" event={"ID":"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6","Type":"ContainerDied","Data":"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177"} Jul 10 20:39:37.577963 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.577933 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" Jul 10 20:39:37.577963 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.577957 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bff4d7977-6smbs" event={"ID":"edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6","Type":"ContainerDied","Data":"67b3fe842e6b6432bc50a770ebcda967ad11be997b9342fc1767a9694a81828b"} Jul 10 20:39:37.578113 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.577976 2566 scope.go:117] "RemoveContainer" containerID="b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177" Jul 10 20:39:37.579114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.579095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtph7" event={"ID":"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2","Type":"ContainerStarted","Data":"936be7f0ec082aa62d4119436712cf3193153e13d302ee80b8cf636748948023"} Jul 10 20:39:37.580270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.580250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" event={"ID":"a2ef422d-131c-4153-8fb2-7eabb17baa8a","Type":"ContainerStarted","Data":"15c6aec9dfcc30d4dc08f583356bfe97d9ba1201bb377e67f15f424f5b8314f8"} Jul 10 20:39:37.586161 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.586143 2566 scope.go:117] "RemoveContainer" containerID="b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177" Jul 10 20:39:37.586434 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:37.586414 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177\": container with ID starting with b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177 not found: ID does not exist" containerID="b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177" Jul 10 20:39:37.586501 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.586441 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177"} err="failed to get container status \"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177\": rpc error: code = NotFound desc = could not find container \"b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177\": container with ID starting with b9742ec2c651d17f6258114b6321f14d6394c35f6827f4d5a371596d23dab177 not found: ID does not exist" Jul 10 20:39:37.599072 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.599054 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:39:37.601873 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.601855 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bff4d7977-6smbs"] Jul 10 20:39:37.777954 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.777918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.778129 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.777982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.778129 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.778008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:37.779055 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.778991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67cd844e-96f3-4352-8975-d84f72dd484c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.780949 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.780898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/67cd844e-96f3-4352-8975-d84f72dd484c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"67cd844e-96f3-4352-8975-d84f72dd484c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:37.781332 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.781305 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a15588c-6e7b-4456-9bf3-d11e304242f9-node-exporter-tls\") pod \"node-exporter-cv28x\" (UID: \"0a15588c-6e7b-4456-9bf3-d11e304242f9\") " pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:37.913160 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:37.913049 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cv28x" Jul 10 20:39:37.926885 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:37.926848 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a15588c_6e7b_4456_9bf3_d11e304242f9.slice/crio-1eaf190e1471b46efcc93af0d61540898e1384523517685b7cd48a9ec8b17a6c WatchSource:0}: Error finding container 1eaf190e1471b46efcc93af0d61540898e1384523517685b7cd48a9ec8b17a6c: Status 404 returned error can't find the container with id 1eaf190e1471b46efcc93af0d61540898e1384523517685b7cd48a9ec8b17a6c Jul 10 20:39:38.058842 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.058693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 20:39:38.155411 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.155381 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-fb47977d8-7w7qc"] Jul 10 20:39:38.160752 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.160728 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.163088 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.163062 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 20:39:38.163573 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.163522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 20:39:38.163777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.163761 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-72ghg\"" Jul 10 20:39:38.163893 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.163843 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 20:39:38.164108 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.164092 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 20:39:38.164499 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.164316 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2ffbq889phabn\"" Jul 10 20:39:38.164499 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.164428 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 20:39:38.171517 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.171431 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fb47977d8-7w7qc"] Jul 10 20:39:38.181541 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.181512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.182037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.182015 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.182317 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.182300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-metrics-client-ca\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.182524 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.182509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tztl4\" (UniqueName: \"kubernetes.io/projected/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-kube-api-access-tztl4\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.182716 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.182700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.182928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.182908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.183113 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.183097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-grpc-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.183298 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.183281 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.227826 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.227722 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 20:39:38.231188 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:38.231159 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67cd844e_96f3_4352_8975_d84f72dd484c.slice/crio-f8cefdd7593f94c807c55c61141a2341da5e1e16808d053e0a067bce7e4f2591 WatchSource:0}: Error finding container f8cefdd7593f94c807c55c61141a2341da5e1e16808d053e0a067bce7e4f2591: Status 404 returned error can't find the container with id f8cefdd7593f94c807c55c61141a2341da5e1e16808d053e0a067bce7e4f2591 Jul 10 20:39:38.284492 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-metrics-client-ca\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tztl4\" (UniqueName: \"kubernetes.io/projected/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-kube-api-access-tztl4\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-grpc-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.284785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.284694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.285731 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.285681 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-metrics-client-ca\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.287888 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.287851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-grpc-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.288543 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.288485 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.288637 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.288582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.288637 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.288618 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.291036 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.291010 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-tls\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.291232 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.291179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.294024 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.294004 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tztl4\" (UniqueName: \"kubernetes.io/projected/8a80bd50-ef2a-47ec-a826-1f25b10da7f8-kube-api-access-tztl4\") pod \"thanos-querier-fb47977d8-7w7qc\" (UID: \"8a80bd50-ef2a-47ec-a826-1f25b10da7f8\") " pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.473783 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.473687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:38.586457 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.586414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cv28x" event={"ID":"0a15588c-6e7b-4456-9bf3-d11e304242f9","Type":"ContainerStarted","Data":"1eaf190e1471b46efcc93af0d61540898e1384523517685b7cd48a9ec8b17a6c"} Jul 10 20:39:38.588152 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:38.588118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"f8cefdd7593f94c807c55c61141a2341da5e1e16808d053e0a067bce7e4f2591"} Jul 10 20:39:39.075529 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.075495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:39.128057 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.127996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:39.221047 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.220976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:39.322236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.322198 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:39.595885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.595850 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6" path="/var/lib/kubelet/pods/edd6fe05-04fc-404a-86c1-6c0a2d6fc0b6/volumes" Jul 10 20:39:39.754528 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:39.754503 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fb47977d8-7w7qc"] Jul 10 20:39:39.925532 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:39.925495 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a80bd50_ef2a_47ec_a826_1f25b10da7f8.slice/crio-e43edf85159db0ab29bb113ac3365105ced86c71a2c99034a4c9993d7a381b13 WatchSource:0}: Error finding container e43edf85159db0ab29bb113ac3365105ced86c71a2c99034a4c9993d7a381b13: Status 404 returned error can't find the container with id e43edf85159db0ab29bb113ac3365105ced86c71a2c99034a4c9993d7a381b13 Jul 10 20:39:40.375732 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.375696 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-566bf96ddb-b5v5z"] Jul 10 20:39:40.378898 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.378876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.381127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381093 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 20:39:40.381127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381111 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 20:39:40.381281 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381169 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-lj5lv\"" Jul 10 20:39:40.381425 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381410 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 20:39:40.381486 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381424 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 20:39:40.381772 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.381760 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e5e45om2l8k6t\"" Jul 10 20:39:40.388839 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.388802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-566bf96ddb-b5v5z"] Jul 10 20:39:40.405684 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-client-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.405838 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-client-certs\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.405838 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-tls\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.405838 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405765 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-metrics-server-audit-profiles\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.405977 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fg6d\" (UniqueName: \"kubernetes.io/projected/28b9c04d-6d28-4785-9994-e73ff48089af-kube-api-access-8fg6d\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.405977 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405943 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.406036 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.405983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/28b9c04d-6d28-4785-9994-e73ff48089af-audit-log\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507282 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-client-certs\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507282 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-tls\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507528 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-metrics-server-audit-profiles\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507528 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fg6d\" (UniqueName: \"kubernetes.io/projected/28b9c04d-6d28-4785-9994-e73ff48089af-kube-api-access-8fg6d\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507528 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507397 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507681 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/28b9c04d-6d28-4785-9994-e73ff48089af-audit-log\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507681 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-client-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.507934 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.507910 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/28b9c04d-6d28-4785-9994-e73ff48089af-audit-log\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.508170 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.508134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.508382 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.508363 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/28b9c04d-6d28-4785-9994-e73ff48089af-metrics-server-audit-profiles\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.510062 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.510039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-client-ca-bundle\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.510389 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.510347 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-client-certs\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.510481 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.510441 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/28b9c04d-6d28-4785-9994-e73ff48089af-secret-metrics-server-tls\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.514954 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.514935 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fg6d\" (UniqueName: \"kubernetes.io/projected/28b9c04d-6d28-4785-9994-e73ff48089af-kube-api-access-8fg6d\") pod \"metrics-server-566bf96ddb-b5v5z\" (UID: \"28b9c04d-6d28-4785-9994-e73ff48089af\") " pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.533401 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.533366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:40.583077 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.583044 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:40.599879 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.599845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtph7" event={"ID":"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2","Type":"ContainerStarted","Data":"55a32faaa21de7a466250afc3d77dc9c440d04d8bd7a08bd8375ff9af4132fa2"} Jul 10 20:39:40.600057 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.599907 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtph7" event={"ID":"2d544536-846f-4bd5-a1af-1d9b4f8bc3a2","Type":"ContainerStarted","Data":"412ad4449a4b1ce8177a5756d265931f2263438a27b2bc8c30123ef49af3420e"} Jul 10 20:39:40.600057 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.599947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:40.602447 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.602422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" event={"ID":"a2ef422d-131c-4153-8fb2-7eabb17baa8a","Type":"ContainerStarted","Data":"5aba9f12a96698faddd99bc3a26ad698deb4c2508008e703d1a89bd5a775ba51"} Jul 10 20:39:40.602644 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.602620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" event={"ID":"a2ef422d-131c-4153-8fb2-7eabb17baa8a","Type":"ContainerStarted","Data":"efec1780a4d90d5fbf3f59bc37746811b6c00aa988ab86a22c425ae4f933ee68"} Jul 10 20:39:40.602765 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.602660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" event={"ID":"a2ef422d-131c-4153-8fb2-7eabb17baa8a","Type":"ContainerStarted","Data":"a3f018b26eb1f6448a51eec984fc656d1764bfe2b99e23a3b490918ede7be47e"} Jul 10 20:39:40.604239 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.604215 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a15588c-6e7b-4456-9bf3-d11e304242f9" containerID="9646ba367dd807db83be01b1e52a3946c296aef1eaa504eb9c8d5163e5ae0e4d" exitCode=0 Jul 10 20:39:40.604352 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.604287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cv28x" event={"ID":"0a15588c-6e7b-4456-9bf3-d11e304242f9","Type":"ContainerDied","Data":"9646ba367dd807db83be01b1e52a3946c296aef1eaa504eb9c8d5163e5ae0e4d"} Jul 10 20:39:40.606110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.606087 2566 generic.go:356] "Generic (PLEG): container finished" podID="67cd844e-96f3-4352-8975-d84f72dd484c" containerID="604eac27517e0c4479449bdeddc0552a62ca6feccf167af90d07fa2ad04895e5" exitCode=0 Jul 10 20:39:40.606212 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.606160 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerDied","Data":"604eac27517e0c4479449bdeddc0552a62ca6feccf167af90d07fa2ad04895e5"} Jul 10 20:39:40.607643 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.607615 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"e43edf85159db0ab29bb113ac3365105ced86c71a2c99034a4c9993d7a381b13"} Jul 10 20:39:40.613239 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.613218 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" event={"ID":"e950d224-6ecb-4395-8124-5df44622e213","Type":"ContainerStarted","Data":"b7d61b77248f1752b9504aedd0eb5f764eeb4e0fe7a6088dcadb3f13bd99bc7c"} Jul 10 20:39:40.616177 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.616125 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xtph7" podStartSLOduration=252.320583096 podStartE2EDuration="4m14.616107666s" totalImagesPullingTime="2.29552457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:40.614830205 +0000 UTC m=+289.498661539" watchObservedRunningTime="2026-07-10 20:39:40.616107666 +0000 UTC m=+289.499939014" Jul 10 20:39:40.631133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.631082 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-bqnp5" podStartSLOduration=2.266329854 podStartE2EDuration="4.63106219s" totalImagesPullingTime="2.364732336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:40.630031468 +0000 UTC m=+289.513862811" watchObservedRunningTime="2026-07-10 20:39:40.63106219 +0000 UTC m=+289.514893535" Jul 10 20:39:40.667758 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.667684 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-q4v9b" podStartSLOduration=2.197781592 podStartE2EDuration="4.667662139s" totalImagesPullingTime="2.469880547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:40.666688206 +0000 UTC m=+289.550519562" watchObservedRunningTime="2026-07-10 20:39:40.667662139 +0000 UTC m=+289.551493483" Jul 10 20:39:40.688788 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.688725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:39:40.854128 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.854094 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-566bf96ddb-b5v5z"] Jul 10 20:39:40.858270 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:40.858236 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b9c04d_6d28_4785_9994_e73ff48089af.slice/crio-18e6936dfd96efad62048ae63112dc8a82dfc90ca35f849dbdbbba918b7f732f WatchSource:0}: Error finding container 18e6936dfd96efad62048ae63112dc8a82dfc90ca35f849dbdbbba918b7f732f: Status 404 returned error can't find the container with id 18e6936dfd96efad62048ae63112dc8a82dfc90ca35f849dbdbbba918b7f732f Jul 10 20:39:40.863192 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.863168 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr"] Jul 10 20:39:40.867586 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.867541 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:40.869685 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.869664 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 10 20:39:40.869787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.869743 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m9tmf\"" Jul 10 20:39:40.874995 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.874910 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:40.876536 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.876481 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr"] Jul 10 20:39:40.914172 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.914139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:40.914350 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.914229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxkp\" (UniqueName: \"kubernetes.io/projected/04d2a6b0-c511-4d27-9610-037a3df3b348-kube-api-access-7nxkp\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:40.932469 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:40.932434 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:41.015363 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.015325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxkp\" (UniqueName: \"kubernetes.io/projected/04d2a6b0-c511-4d27-9610-037a3df3b348-kube-api-access-7nxkp\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.015535 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.015420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.015535 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:41.015507 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 10 20:39:41.015667 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:39:41.015569 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert podName:04d2a6b0-c511-4d27-9610-037a3df3b348 nodeName:}" failed. No retries permitted until 2026-07-10 20:39:41.515547497 +0000 UTC m=+290.399378818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-p7rvr" (UID: "04d2a6b0-c511-4d27-9610-037a3df3b348") : secret "monitoring-plugin-cert" not found Jul 10 20:39:41.024114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.024085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxkp\" (UniqueName: \"kubernetes.io/projected/04d2a6b0-c511-4d27-9610-037a3df3b348-kube-api-access-7nxkp\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.180130 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.180052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:41.337950 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.337916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:41.386870 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.386836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:41.520797 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.520758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.524866 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.524837 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/04d2a6b0-c511-4d27-9610-037a3df3b348-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-p7rvr\" (UID: \"04d2a6b0-c511-4d27-9610-037a3df3b348\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.619527 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.619491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" event={"ID":"28b9c04d-6d28-4785-9994-e73ff48089af","Type":"ContainerStarted","Data":"18e6936dfd96efad62048ae63112dc8a82dfc90ca35f849dbdbbba918b7f732f"} Jul 10 20:39:41.622456 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.622315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cv28x" event={"ID":"0a15588c-6e7b-4456-9bf3-d11e304242f9","Type":"ContainerStarted","Data":"3238f4d64ae90bacef97da737c73fa591a8a294d40a0ccf4ea8fcd0fb5bc40f5"} Jul 10 20:39:41.622596 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.622514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cv28x" event={"ID":"0a15588c-6e7b-4456-9bf3-d11e304242f9","Type":"ContainerStarted","Data":"c9049dad86821c3ac693227c0af53cba0a57277a97fc382152932632b6c9a1fb"} Jul 10 20:39:41.642091 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.641973 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cv28x" podStartSLOduration=3.52654792 podStartE2EDuration="5.641950348s" totalImagesPullingTime="1.997849648s" totalInitContainerRuntime="117.55278ms" isStatefulPod=true podCreationTimestamp="2026-07-10 20:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:41.641124163 +0000 UTC m=+290.524955507" watchObservedRunningTime="2026-07-10 20:39:41.641950348 +0000 UTC m=+290.525781693" Jul 10 20:39:41.780103 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.779937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:41.859257 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.859221 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:41.943015 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:41.942974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:42.341039 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:42.340986 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:42.566200 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:42.566168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:42.666190 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:42.666107 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:43.377930 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.377903 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr"] Jul 10 20:39:43.381292 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:39:43.381250 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d2a6b0_c511_4d27_9610_037a3df3b348.slice/crio-c86760095230668661f2f730831808de161e34906456e024c611ea5c9723075c WatchSource:0}: Error finding container c86760095230668661f2f730831808de161e34906456e024c611ea5c9723075c: Status 404 returned error can't find the container with id c86760095230668661f2f730831808de161e34906456e024c611ea5c9723075c Jul 10 20:39:43.632006 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.631978 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"5ed4efc23f9e1762db54ff9f0f4cac361604671caca0361dcb4860ec4f6adcf7"} Jul 10 20:39:43.632366 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.632045 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"7f46eac878ffb438deb75947122d9c9abfc9e9ba75c7a5a9e793594560b0e889"} Jul 10 20:39:43.634828 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.634638 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"f8edd5c6e9da34f8ff2df39ccebd7046a6e730570dc2d63566c43284f9848b5f"} Jul 10 20:39:43.634925 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.634876 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"eb18199dce3a8725edc70d96f25af60cb479df76b55117a7060ed5a8d6f55cd3"} Jul 10 20:39:43.634925 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.634919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"7707f931b60f2ca06bf18918f03dcb753e6e01824f0feed61b8e6d57549dbb7d"} Jul 10 20:39:43.636420 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.636389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" event={"ID":"28b9c04d-6d28-4785-9994-e73ff48089af","Type":"ContainerStarted","Data":"001fb1dba51291b0316663a922ba458b9a767de281b8cf4c00dfe3eee27dcecc"} Jul 10 20:39:43.637821 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.637790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" event={"ID":"04d2a6b0-c511-4d27-9610-037a3df3b348","Type":"ContainerStarted","Data":"c86760095230668661f2f730831808de161e34906456e024c611ea5c9723075c"} Jul 10 20:39:43.660144 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:43.660094 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" podStartSLOduration=1.2768536369999999 podStartE2EDuration="3.660073327s" totalImagesPullingTime="2.38321969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:43.657697 +0000 UTC m=+292.541528367" watchObservedRunningTime="2026-07-10 20:39:43.660073327 +0000 UTC m=+292.543904671" Jul 10 20:39:44.646336 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:44.646301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"cb9f607c5ec08cca64c328b02f50b868f18adacf72127518ff0a6c38fb62275c"} Jul 10 20:39:44.646704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:44.646390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"9e2990a4f331a9bbc1e1215df27e64e89ab04350cdefc4f94bd524350bd71012"} Jul 10 20:39:44.646704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:44.646431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"3435dda430128fcb41f91f8533f3647ce7351b75891af66852953603ca05fec9"} Jul 10 20:39:45.653262 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.653181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"67cd844e-96f3-4352-8975-d84f72dd484c","Type":"ContainerStarted","Data":"75f8045e0688d39acc73e06b9907f6a239b8f6708147c44d21cb075b196875f4"} Jul 10 20:39:45.656113 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.656084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"14d1ba908eb86db54e5f1714f1206860918a305fe2e97d51d516157977512f65"} Jul 10 20:39:45.656214 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.656153 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"7455cd00090d51bb78dcbbd7c3120d2ab2b0687eac662e6601cf0a3d3573cf6a"} Jul 10 20:39:45.656214 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.656199 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:45.656323 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.656232 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" event={"ID":"8a80bd50-ef2a-47ec-a826-1f25b10da7f8","Type":"ContainerStarted","Data":"c24f86a1d3efc6aa9d020e6a3f016fc18ea269ae0b5d7414ce0be3c91dde3716"} Jul 10 20:39:45.657439 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.657414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" event={"ID":"04d2a6b0-c511-4d27-9610-037a3df3b348","Type":"ContainerStarted","Data":"ae7d0e873301f094a5e8639035b4a60620b576764855c3b725bdc49687a5d165"} Jul 10 20:39:45.657626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.657610 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:45.662424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.662406 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" Jul 10 20:39:45.680526 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.680479 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8164544769999997 podStartE2EDuration="8.680460122s" totalImagesPullingTime="5.829227759s" totalInitContainerRuntime="34.777886ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:45.678617264 +0000 UTC m=+294.562448618" watchObservedRunningTime="2026-07-10 20:39:45.680460122 +0000 UTC m=+294.564291466" Jul 10 20:39:45.700979 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:45.700923 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" podStartSLOduration=2.685574404 podStartE2EDuration="7.700902712s" totalImagesPullingTime="5.015328308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:45.698969657 +0000 UTC m=+294.582801000" watchObservedRunningTime="2026-07-10 20:39:45.700902712 +0000 UTC m=+294.584734055" Jul 10 20:39:46.696459 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:46.696422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:46.797246 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:46.797212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:50.624932 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:50.624899 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xtph7" Jul 10 20:39:50.642504 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:50.642458 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-p7rvr" podStartSLOduration=8.711644865 podStartE2EDuration="10.642442622s" totalImagesPullingTime="1.930797757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:45.715068273 +0000 UTC m=+294.598899617" watchObservedRunningTime="2026-07-10 20:39:50.642442622 +0000 UTC m=+299.526273964" Jul 10 20:39:50.781682 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:50.781648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:50.882401 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:50.882336 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:51.474546 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.474518 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:39:51.476294 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.476271 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:39:51.479664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.479646 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:39:51.480921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.480903 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:39:51.485838 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.485823 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 20:39:51.669102 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:51.668974 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-fb47977d8-7w7qc" Jul 10 20:39:53.074861 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:53.074834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:53.171510 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:53.171476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:53.271654 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:53.271616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:53.958940 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:53.958908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:54.058020 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:54.057984 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:55.767826 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:55.767786 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:39:55.865824 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:39:55.865790 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:00.689485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:00.689452 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:40:00.689893 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:00.689509 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:40:02.948185 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:02.948147 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:03.003266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:03.003226 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:11.700557 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:11.700518 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:11.748725 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:11.748696 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:20.695018 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:20.694984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:40:20.698855 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:20.698835 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-566bf96ddb-b5v5z" Jul 10 20:40:49.215780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:49.215705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:49.271513 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:49.271478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:55.328365 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:55.328333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:40:55.748931 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:40:55.748902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:00.402384 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.402331 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf"] Jul 10 20:41:00.405858 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.405827 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.408628 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408599 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 20:41:00.408628 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408618 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 20:41:00.408828 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408688 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 20:41:00.408908 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408893 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 20:41:00.408989 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6mcbs\"" Jul 10 20:41:00.409065 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.408994 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 20:41:00.414042 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.413935 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 20:41:00.419660 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.419630 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf"] Jul 10 20:41:00.577511 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2md9\" (UniqueName: \"kubernetes.io/projected/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-kube-api-access-r2md9\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577511 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-serving-certs-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-metrics-client-ca\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-federate-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577608 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577642 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.577728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.577674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.678785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-federate-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.678785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678732 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.678785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.678785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679178 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679178 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2md9\" (UniqueName: \"kubernetes.io/projected/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-kube-api-access-r2md9\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679178 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-serving-certs-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679178 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.678995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-metrics-client-ca\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679760 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.679733 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-serving-certs-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.679908 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.679804 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-metrics-client-ca\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.680053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.680030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.681465 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.681446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.681739 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.681720 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.681823 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.681737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-telemeter-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.681871 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.681825 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-federate-client-tls\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.686756 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.686736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2md9\" (UniqueName: \"kubernetes.io/projected/ccd9cf00-e4fc-43d4-9cd7-01d0211fc362-kube-api-access-r2md9\") pod \"telemeter-client-9c47c6d5b-4tbdf\" (UID: \"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362\") " pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.716832 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.716797 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" Jul 10 20:41:00.844900 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.844877 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf"] Jul 10 20:41:00.847243 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:41:00.847216 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd9cf00_e4fc_43d4_9cd7_01d0211fc362.slice/crio-78cad3771b55f684d67f52e37a27ab1a5dcff0088a2c09b962d34ff514834624 WatchSource:0}: Error finding container 78cad3771b55f684d67f52e37a27ab1a5dcff0088a2c09b962d34ff514834624: Status 404 returned error can't find the container with id 78cad3771b55f684d67f52e37a27ab1a5dcff0088a2c09b962d34ff514834624 Jul 10 20:41:00.849131 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.849114 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:41:00.952184 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:00.952091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" event={"ID":"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362","Type":"ContainerStarted","Data":"78cad3771b55f684d67f52e37a27ab1a5dcff0088a2c09b962d34ff514834624"} Jul 10 20:41:02.236256 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:02.236212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:02.289854 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:02.289804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:03.967545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:03.967511 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" event={"ID":"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362","Type":"ContainerStarted","Data":"827e08c8696c2d0d244d73a2d8a7fcaacce91af24c54b9a221005f1ff7a392a7"} Jul 10 20:41:03.967974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:03.967561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" event={"ID":"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362","Type":"ContainerStarted","Data":"db561fe85429e08d758789be3362540f7799c065d4627ef87f7581df2eccb322"} Jul 10 20:41:03.967974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:03.967581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" event={"ID":"ccd9cf00-e4fc-43d4-9cd7-01d0211fc362","Type":"ContainerStarted","Data":"b15e96f144a1b8d8089ad765b988675b6e2377133e3835a65c3ad89c6a1a868b"} Jul 10 20:41:03.988440 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:03.988383 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9c47c6d5b-4tbdf" podStartSLOduration=1.542909945 podStartE2EDuration="3.988361986s" totalImagesPullingTime="2.445452041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:03.986761428 +0000 UTC m=+372.870592792" watchObservedRunningTime="2026-07-10 20:41:03.988361986 +0000 UTC m=+372.872193330" Jul 10 20:41:04.744523 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:04.744487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:04.845665 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:04.845636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:04.896967 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:04.896939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:05.704113 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:05.704081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:10.548081 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:10.548044 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:10.596197 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:10.596168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:14.233048 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:14.233010 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:14.284710 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:14.284676 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:14.839038 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:14.839000 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:16.217442 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:16.217410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:22.548047 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:22.548013 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:22.599351 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:22.599320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:25.408235 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:25.408202 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:25.461111 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:25.461075 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:25.512833 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:25.512778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:25.615603 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:25.615563 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:25.666559 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:25.666486 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:26.703919 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:26.703873 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:26.804575 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:26.804540 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:53.761654 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:53.761625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:41:53.861284 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:41:53.861251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.094089 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:14.094006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:14.151916 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:14.151888 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:20.383304 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:20.383268 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:21.205919 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:21.205889 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:22.716123 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:22.716090 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:24.000157 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:24.000119 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:24.054463 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:24.054429 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:25.036750 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:25.036719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:25.148622 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:25.148585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:25.903164 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:25.903132 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:25.991121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:25.991089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:28.092892 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:28.092860 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:28.289039 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:28.289009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:30.627545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:30.627492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:30.677662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:30.677628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:36.294497 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:36.294450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:40.004071 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:40.004031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:40.104425 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:40.104389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:41.718028 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.717997 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq"] Jul 10 20:42:41.721281 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.721263 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.723530 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.723510 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 20:42:41.723630 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.723556 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 20:42:41.724392 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.724379 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-mwf2w\"" Jul 10 20:42:41.729126 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.729106 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq"] Jul 10 20:42:41.851342 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.851304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jjs9\" (UniqueName: \"kubernetes.io/projected/ac3052ea-2573-480e-8ce9-ef3685353b68-kube-api-access-6jjs9\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.851538 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.851392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac3052ea-2573-480e-8ce9-ef3685353b68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.952721 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.952689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jjs9\" (UniqueName: \"kubernetes.io/projected/ac3052ea-2573-480e-8ce9-ef3685353b68-kube-api-access-6jjs9\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.952853 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.952748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac3052ea-2573-480e-8ce9-ef3685353b68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.953173 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.953155 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac3052ea-2573-480e-8ce9-ef3685353b68-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:41.960276 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:41.960254 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jjs9\" (UniqueName: \"kubernetes.io/projected/ac3052ea-2573-480e-8ce9-ef3685353b68-kube-api-access-6jjs9\") pod \"openshift-lws-operator-fd8ccff4c-7g2tq\" (UID: \"ac3052ea-2573-480e-8ce9-ef3685353b68\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:42.031309 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:42.031273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" Jul 10 20:42:42.157354 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:42.157281 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq"] Jul 10 20:42:42.159764 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:42:42.159735 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac3052ea_2573_480e_8ce9_ef3685353b68.slice/crio-9905baaa1e2022ed016fd6a59e052c870968a48a01f06b4334db8583ad1e072a WatchSource:0}: Error finding container 9905baaa1e2022ed016fd6a59e052c870968a48a01f06b4334db8583ad1e072a: Status 404 returned error can't find the container with id 9905baaa1e2022ed016fd6a59e052c870968a48a01f06b4334db8583ad1e072a Jul 10 20:42:42.354899 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:42.354800 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" event={"ID":"ac3052ea-2573-480e-8ce9-ef3685353b68","Type":"ContainerStarted","Data":"9905baaa1e2022ed016fd6a59e052c870968a48a01f06b4334db8583ad1e072a"} Jul 10 20:42:45.370120 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:45.370080 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" event={"ID":"ac3052ea-2573-480e-8ce9-ef3685353b68","Type":"ContainerStarted","Data":"91949b49e541e44c06cdf3ffdcf431e080d81be21e5e7f9a4450af9c4d15db82"} Jul 10 20:42:45.385254 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:45.385201 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7g2tq" podStartSLOduration=1.941176006 podStartE2EDuration="4.385187479s" totalImagesPullingTime="2.444011473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:42:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:42:45.384190224 +0000 UTC m=+474.268021588" watchObservedRunningTime="2026-07-10 20:42:45.385187479 +0000 UTC m=+474.269018822" Jul 10 20:42:46.034921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:46.034887 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:46.133035 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:46.132999 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:46.234380 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:46.234346 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:46.333921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:46.333835 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:46.385363 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:46.385328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:48.273563 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:48.273536 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:56.489828 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:56.489789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:42:56.600329 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:42:56.600293 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:03.598419 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:03.598388 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:03.713569 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:03.713535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.223403 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.223364 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.422388 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.422349 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.529507 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.529474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.543968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.543933 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9"] Jul 10 20:43:04.551317 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.551290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.554014 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.553989 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 20:43:04.554167 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.554062 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 20:43:04.554167 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.554104 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 20:43:04.554167 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.554116 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 20:43:04.554552 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.554539 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t5qrt\"" Jul 10 20:43:04.574844 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.574802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9"] Jul 10 20:43:04.653593 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.653561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-tmp\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.654133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.653720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.654133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.653774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.654133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.653878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4spv\" (UniqueName: \"kubernetes.io/projected/c0074893-134d-41c6-81b6-090b0a80cbb2-kube-api-access-j4spv\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.654133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.654004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-manifests\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755100 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755100 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4spv\" (UniqueName: \"kubernetes.io/projected/c0074893-134d-41c6-81b6-090b0a80cbb2-kube-api-access-j4spv\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-manifests\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-tmp\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.755689 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.755656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-manifests\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.757503 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.757477 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0074893-134d-41c6-81b6-090b0a80cbb2-tmp\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.757724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.757706 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-webhook-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.757799 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.757713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c0074893-134d-41c6-81b6-090b0a80cbb2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.763412 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.763388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4spv\" (UniqueName: \"kubernetes.io/projected/c0074893-134d-41c6-81b6-090b0a80cbb2-kube-api-access-j4spv\") pod \"opendatahub-operator-controller-manager-d5798c7df-xk6j9\" (UID: \"c0074893-134d-41c6-81b6-090b0a80cbb2\") " pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.763707 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.763685 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.862088 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.862001 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:04.863757 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.863731 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.966039 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.966007 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:04.997963 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:04.997943 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9"] Jul 10 20:43:05.000764 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:05.000731 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0074893_134d_41c6_81b6_090b0a80cbb2.slice/crio-e3c38b49a705134733f19af3f21f0e2f104eb54258b065f84408598eda66d1ef WatchSource:0}: Error finding container e3c38b49a705134733f19af3f21f0e2f104eb54258b065f84408598eda66d1ef: Status 404 returned error can't find the container with id e3c38b49a705134733f19af3f21f0e2f104eb54258b065f84408598eda66d1ef Jul 10 20:43:05.066751 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:05.066719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:05.166549 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:05.166464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:05.265223 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:05.265180 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:05.456338 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:05.456253 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" event={"ID":"c0074893-134d-41c6-81b6-090b0a80cbb2","Type":"ContainerStarted","Data":"e3c38b49a705134733f19af3f21f0e2f104eb54258b065f84408598eda66d1ef"} Jul 10 20:43:07.769359 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:07.769315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:08.474007 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:08.473932 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" event={"ID":"c0074893-134d-41c6-81b6-090b0a80cbb2","Type":"ContainerStarted","Data":"397604132f996d71fb14717a823e080e01ecc1f41aaf3b62f4472feb695b2e8b"} Jul 10 20:43:09.480465 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:09.480436 2566 generic.go:356] "Generic (PLEG): container finished" podID="c0074893-134d-41c6-81b6-090b0a80cbb2" containerID="397604132f996d71fb14717a823e080e01ecc1f41aaf3b62f4472feb695b2e8b" exitCode=0 Jul 10 20:43:09.480862 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:09.480504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" event={"ID":"c0074893-134d-41c6-81b6-090b0a80cbb2","Type":"ContainerDied","Data":"397604132f996d71fb14717a823e080e01ecc1f41aaf3b62f4472feb695b2e8b"} Jul 10 20:43:10.487437 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:10.487402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" event={"ID":"c0074893-134d-41c6-81b6-090b0a80cbb2","Type":"ContainerStarted","Data":"5dbb815caaf4efa2a710a38beb935a82e551410b2b7e42243ee90aab3197f36a"} Jul 10 20:43:10.487949 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:10.487629 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:10.509883 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:10.509793 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" podStartSLOduration=1.959893064 podStartE2EDuration="6.509770191s" totalImagesPullingTime="3.449386829s" totalInitContainerRuntime="1.100490298s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:10.508579172 +0000 UTC m=+499.392410515" watchObservedRunningTime="2026-07-10 20:43:10.509770191 +0000 UTC m=+499.393601539" Jul 10 20:43:21.494670 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:21.494641 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d5798c7df-xk6j9" Jul 10 20:43:23.220035 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.219996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:23.329993 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.329953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:23.892435 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.892399 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bkqfb"] Jul 10 20:43:23.895059 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.895036 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:23.898255 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.898236 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zcj2l\"" Jul 10 20:43:23.899604 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.899587 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 20:43:23.904756 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.904733 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bkqfb"] Jul 10 20:43:23.922263 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:23.922221 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.026391 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.026343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.029992 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.029972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zrp\" (UniqueName: \"kubernetes.io/projected/b20a0b31-9eb3-4f6a-9803-d4991413859f-kube-api-access-74zrp\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.030106 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.030013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.122763 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.122722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.130928 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.130899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zrp\" (UniqueName: \"kubernetes.io/projected/b20a0b31-9eb3-4f6a-9803-d4991413859f-kube-api-access-74zrp\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.131092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.130959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.131151 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:24.131109 2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 20:43:24.131201 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:24.131165 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert podName:b20a0b31-9eb3-4f6a-9803-d4991413859f nodeName:}" failed. No retries permitted until 2026-07-10 20:43:24.631147425 +0000 UTC m=+513.514978745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert") pod "odh-model-controller-7998c67dc4-bkqfb" (UID: "b20a0b31-9eb3-4f6a-9803-d4991413859f") : secret "odh-model-controller-webhook-cert" not found Jul 10 20:43:24.139279 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.139258 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zrp\" (UniqueName: \"kubernetes.io/projected/b20a0b31-9eb3-4f6a-9803-d4991413859f-kube-api-access-74zrp\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.223167 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.223056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.361131 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.361100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.422638 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.422597 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-db5gc"] Jul 10 20:43:24.425331 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.425312 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.427551 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.427537 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 20:43:24.427801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.427789 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8x468\"" Jul 10 20:43:24.436016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.435995 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-db5gc"] Jul 10 20:43:24.534458 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.534429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce32507d-7fec-4c1b-b508-42abf660f61a-tls-certs\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.534628 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.534485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bp8\" (UniqueName: \"kubernetes.io/projected/ce32507d-7fec-4c1b-b508-42abf660f61a-kube-api-access-b8bp8\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.575117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.575062 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:24.635327 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.635291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce32507d-7fec-4c1b-b508-42abf660f61a-tls-certs\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.635520 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.635363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bp8\" (UniqueName: \"kubernetes.io/projected/ce32507d-7fec-4c1b-b508-42abf660f61a-kube-api-access-b8bp8\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.635520 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.635403 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.637745 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.637716 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b20a0b31-9eb3-4f6a-9803-d4991413859f-cert\") pod \"odh-model-controller-7998c67dc4-bkqfb\" (UID: \"b20a0b31-9eb3-4f6a-9803-d4991413859f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.637895 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.637874 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce32507d-7fec-4c1b-b508-42abf660f61a-tls-certs\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.643549 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.643518 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bp8\" (UniqueName: \"kubernetes.io/projected/ce32507d-7fec-4c1b-b508-42abf660f61a-kube-api-access-b8bp8\") pod \"model-serving-api-54d89bd5fc-db5gc\" (UID: \"ce32507d-7fec-4c1b-b508-42abf660f61a\") " pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.737095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.737060 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:24.805367 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.805331 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:24.876127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.876097 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-db5gc"] Jul 10 20:43:24.879025 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:24.878995 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce32507d_7fec_4c1b_b508_42abf660f61a.slice/crio-66117b4ce54fdf848d5a1df9e242cd6a2f52fc9e2892f88fb8758108b6387287 WatchSource:0}: Error finding container 66117b4ce54fdf848d5a1df9e242cd6a2f52fc9e2892f88fb8758108b6387287: Status 404 returned error can't find the container with id 66117b4ce54fdf848d5a1df9e242cd6a2f52fc9e2892f88fb8758108b6387287 Jul 10 20:43:24.941241 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:24.941216 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bkqfb"] Jul 10 20:43:24.942897 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:24.942872 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20a0b31_9eb3_4f6a_9803_d4991413859f.slice/crio-a96edef8570e6f69fd007291ed57309a178909c7cae60d560a911d0756f78036 WatchSource:0}: Error finding container a96edef8570e6f69fd007291ed57309a178909c7cae60d560a911d0756f78036: Status 404 returned error can't find the container with id a96edef8570e6f69fd007291ed57309a178909c7cae60d560a911d0756f78036 Jul 10 20:43:25.267759 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:25.267730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:25.319545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:25.319514 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:25.555682 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:25.555604 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" event={"ID":"b20a0b31-9eb3-4f6a-9803-d4991413859f","Type":"ContainerStarted","Data":"a96edef8570e6f69fd007291ed57309a178909c7cae60d560a911d0756f78036"} Jul 10 20:43:25.557027 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:25.556993 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" event={"ID":"ce32507d-7fec-4c1b-b508-42abf660f61a","Type":"ContainerStarted","Data":"66117b4ce54fdf848d5a1df9e242cd6a2f52fc9e2892f88fb8758108b6387287"} Jul 10 20:43:28.578589 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.578555 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" event={"ID":"b20a0b31-9eb3-4f6a-9803-d4991413859f","Type":"ContainerStarted","Data":"47eab7c19f0906e83943a12b23622c941c2cf0b282704e217d3254d8c5d429d2"} Jul 10 20:43:28.579093 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.578672 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:28.580350 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.580324 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" event={"ID":"ce32507d-7fec-4c1b-b508-42abf660f61a","Type":"ContainerStarted","Data":"2032fc4cb7113bfc0a74564a51e8f299d1b4814d95c9d77612b48ecec8eb70ea"} Jul 10 20:43:28.580483 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.580389 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:28.596547 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.596504 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" podStartSLOduration=2.519430115 podStartE2EDuration="5.596491852s" totalImagesPullingTime="3.077061737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:28.594731298 +0000 UTC m=+517.478562641" watchObservedRunningTime="2026-07-10 20:43:28.596491852 +0000 UTC m=+517.480323206" Jul 10 20:43:28.610996 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:28.610942 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" podStartSLOduration=1.472560739 podStartE2EDuration="4.610926597s" totalImagesPullingTime="3.138365858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:28.610062725 +0000 UTC m=+517.493894069" watchObservedRunningTime="2026-07-10 20:43:28.610926597 +0000 UTC m=+517.494757941" Jul 10 20:43:29.044830 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.044764 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:29.055455 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.055427 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc"] Jul 10 20:43:29.058469 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.058451 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.060589 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.060569 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 20:43:29.060706 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.060670 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 20:43:29.069786 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.069762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc"] Jul 10 20:43:29.181219 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.181177 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04e40346-1e41-42cf-9d52-b04c72845cba-tmp\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.181219 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.181220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/04e40346-1e41-42cf-9d52-b04c72845cba-kube-api-access-v2rlj\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.181467 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.181240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.282578 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.282540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04e40346-1e41-42cf-9d52-b04c72845cba-tmp\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.282578 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.282590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/04e40346-1e41-42cf-9d52-b04c72845cba-kube-api-access-v2rlj\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.282847 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.282609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.282847 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:29.282732 2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 20:43:29.282847 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:29.282800 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs podName:04e40346-1e41-42cf-9d52-b04c72845cba nodeName:}" failed. No retries permitted until 2026-07-10 20:43:29.782780501 +0000 UTC m=+518.666611824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs") pod "kube-auth-proxy-7d744644f5-xbcjc" (UID: "04e40346-1e41-42cf-9d52-b04c72845cba") : secret "kube-auth-proxy-tls" not found Jul 10 20:43:29.284863 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.284844 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04e40346-1e41-42cf-9d52-b04c72845cba-tmp\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.293213 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.293186 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2rlj\" (UniqueName: \"kubernetes.io/projected/04e40346-1e41-42cf-9d52-b04c72845cba-kube-api-access-v2rlj\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.461435 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.461353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:29.788084 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.788046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.790568 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.790533 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04e40346-1e41-42cf-9d52-b04c72845cba-tls-certs\") pod \"kube-auth-proxy-7d744644f5-xbcjc\" (UID: \"04e40346-1e41-42cf-9d52-b04c72845cba\") " pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:29.969444 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:29.969408 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" Jul 10 20:43:30.104215 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.104058 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:30.110065 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.110042 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.112724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.112257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 20:43:30.112724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.112413 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h46w4\"" Jul 10 20:43:30.112724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.112257 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 20:43:30.112724 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.112268 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 20:43:30.113013 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.112751 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 20:43:30.120065 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.119829 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:30.134252 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.134227 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc"] Jul 10 20:43:30.136746 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.136719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:30.138528 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:30.138503 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e40346_1e41_42cf_9d52_b04c72845cba.slice/crio-642ab17cb789adef5aa14009a52ac2431598021fd57a47630b9f019936b28ee4 WatchSource:0}: Error finding container 642ab17cb789adef5aa14009a52ac2431598021fd57a47630b9f019936b28ee4: Status 404 returned error can't find the container with id 642ab17cb789adef5aa14009a52ac2431598021fd57a47630b9f019936b28ee4 Jul 10 20:43:30.232002 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.231973 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:30.291695 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291664 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.291885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.291885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.291885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9f79\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.291885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291822 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.292038 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.292038 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.291906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.392960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393053 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9f79\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393136 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393161 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.393795 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.393773 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.396567 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.396541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.396686 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.396583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.396686 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.396655 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.396755 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.396699 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.402753 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.402721 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9f79\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.402888 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.402772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token\") pod \"istiod-openshift-gateway-7574459656-pzmcs\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.423649 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.423624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:30.565362 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.565331 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:30.569899 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:30.569873 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ca0227_498f_448a_a7ee_a27c55338904.slice/crio-812e86621e67ab4c94dde99c917847c3d5b85bc5d12b2a6d39887b54520aba84 WatchSource:0}: Error finding container 812e86621e67ab4c94dde99c917847c3d5b85bc5d12b2a6d39887b54520aba84: Status 404 returned error can't find the container with id 812e86621e67ab4c94dde99c917847c3d5b85bc5d12b2a6d39887b54520aba84 Jul 10 20:43:30.591509 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.591481 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" event={"ID":"36ca0227-498f-448a-a7ee-a27c55338904","Type":"ContainerStarted","Data":"812e86621e67ab4c94dde99c917847c3d5b85bc5d12b2a6d39887b54520aba84"} Jul 10 20:43:30.592685 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:30.592663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" event={"ID":"04e40346-1e41-42cf-9d52-b04c72845cba","Type":"ContainerStarted","Data":"642ab17cb789adef5aa14009a52ac2431598021fd57a47630b9f019936b28ee4"} Jul 10 20:43:31.249735 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:31.249696 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:33.609651 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:33.609578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" event={"ID":"04e40346-1e41-42cf-9d52-b04c72845cba","Type":"ContainerStarted","Data":"c02178162ea8e16251c4ce380200fbdeeee9a2b5d5a8c4ba659ae58746864a87"} Jul 10 20:43:33.630358 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:33.630291 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d744644f5-xbcjc" podStartSLOduration=1.518947571 podStartE2EDuration="4.630271337s" totalImagesPullingTime="3.111323766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:33.626776245 +0000 UTC m=+522.510607600" watchObservedRunningTime="2026-07-10 20:43:33.630271337 +0000 UTC m=+522.514102681" Jul 10 20:43:34.569943 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.569906 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:34.627273 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.627242 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm"] Jul 10 20:43:34.630634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.630614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.633058 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.633034 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 20:43:34.633162 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.633064 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5ngzc\"" Jul 10 20:43:34.633421 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.633400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.633462 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.633430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk97z\" (UniqueName: \"kubernetes.io/projected/7f6a6fdc-3484-4755-9ac9-f01810bcd773-kube-api-access-mk97z\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.644626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.644600 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm"] Jul 10 20:43:34.648552 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.648533 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm"] Jul 10 20:43:34.648651 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.648611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.651130 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.651112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 20:43:34.651623 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.651602 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vdq8t\"" Jul 10 20:43:34.664234 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.664207 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm"] Jul 10 20:43:34.673027 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.672999 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:34.734114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.734085 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.734114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.734118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk97z\" (UniqueName: \"kubernetes.io/projected/7f6a6fdc-3484-4755-9ac9-f01810bcd773-kube-api-access-mk97z\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.734310 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.734144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4s6z\" (UniqueName: \"kubernetes.io/projected/bdac9a85-5cf2-427b-878f-da31494bcb49-kube-api-access-d4s6z\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.734310 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.734198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.734310 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:34.734227 2566 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 20:43:34.734310 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:34.734281 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert podName:7f6a6fdc-3484-4755-9ac9-f01810bcd773 nodeName:}" failed. No retries permitted until 2026-07-10 20:43:35.234263664 +0000 UTC m=+524.118094986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert") pod "kserve-controller-manager-d6f4b65b8-4dpkm" (UID: "7f6a6fdc-3484-4755-9ac9-f01810bcd773") : secret "kserve-webhook-server-cert" not found Jul 10 20:43:34.743017 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.742994 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk97z\" (UniqueName: \"kubernetes.io/projected/7f6a6fdc-3484-4755-9ac9-f01810bcd773-kube-api-access-mk97z\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:34.835224 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.835149 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.835356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.835263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4s6z\" (UniqueName: \"kubernetes.io/projected/bdac9a85-5cf2-427b-878f-da31494bcb49-kube-api-access-d4s6z\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.835356 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:34.835315 2566 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 20:43:34.835447 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:34.835373 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert podName:bdac9a85-5cf2-427b-878f-da31494bcb49 nodeName:}" failed. No retries permitted until 2026-07-10 20:43:35.335359279 +0000 UTC m=+524.219190600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert") pod "llmisvc-controller-manager-777c6bd7d5-4rtrm" (UID: "bdac9a85-5cf2-427b-878f-da31494bcb49") : secret "llmisvc-webhook-server-cert" not found Jul 10 20:43:34.846424 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.846393 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4s6z\" (UniqueName: \"kubernetes.io/projected/bdac9a85-5cf2-427b-878f-da31494bcb49-kube-api-access-d4s6z\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:34.884911 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:34.884883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:35.216377 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.216296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:35.238497 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.238470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:35.240841 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.240804 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f6a6fdc-3484-4755-9ac9-f01810bcd773-cert\") pod \"kserve-controller-manager-d6f4b65b8-4dpkm\" (UID: \"7f6a6fdc-3484-4755-9ac9-f01810bcd773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:35.339012 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.338974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:35.341371 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.341348 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdac9a85-5cf2-427b-878f-da31494bcb49-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4rtrm\" (UID: \"bdac9a85-5cf2-427b-878f-da31494bcb49\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:35.540388 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.540354 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:35.558675 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.558644 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:35.682996 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.682911 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm"] Jul 10 20:43:35.685183 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:35.685156 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6a6fdc_3484_4755_9ac9_f01810bcd773.slice/crio-47e4324355e94b5e6c6bbbb006c5881fbbbbf115d3076dd8c2a2fa73218a7d7b WatchSource:0}: Error finding container 47e4324355e94b5e6c6bbbb006c5881fbbbbf115d3076dd8c2a2fa73218a7d7b: Status 404 returned error can't find the container with id 47e4324355e94b5e6c6bbbb006c5881fbbbbf115d3076dd8c2a2fa73218a7d7b Jul 10 20:43:35.717519 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.717498 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm"] Jul 10 20:43:35.719868 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:35.719839 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbdac9a85_5cf2_427b_878f_da31494bcb49.slice/crio-8a8498027e68427233d09219d1027cdc9bac54b3c4493ddeebd947b273bfba48 WatchSource:0}: Error finding container 8a8498027e68427233d09219d1027cdc9bac54b3c4493ddeebd947b273bfba48: Status 404 returned error can't find the container with id 8a8498027e68427233d09219d1027cdc9bac54b3c4493ddeebd947b273bfba48 Jul 10 20:43:35.886387 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.886304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:35.981675 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:35.981643 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:36.041103 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:36.041077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:36.632407 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:36.632329 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" event={"ID":"7f6a6fdc-3484-4755-9ac9-f01810bcd773","Type":"ContainerStarted","Data":"47e4324355e94b5e6c6bbbb006c5881fbbbbf115d3076dd8c2a2fa73218a7d7b"} Jul 10 20:43:36.633712 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:36.633675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" event={"ID":"bdac9a85-5cf2-427b-878f-da31494bcb49","Type":"ContainerStarted","Data":"8a8498027e68427233d09219d1027cdc9bac54b3c4493ddeebd947b273bfba48"} Jul 10 20:43:36.788390 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:36.788354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:36.889335 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:36.889243 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:37.551826 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.551782 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks"] Jul 10 20:43:37.555448 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.555425 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.558188 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558136 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fwxnq\"" Jul 10 20:43:37.558695 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.558856 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.558856 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5d156fd-93b2-4e76-a510-e2e36422448d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.558856 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.558856 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.559127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.558967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.559127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.559027 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26qn\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-kube-api-access-v26qn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.559127 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.559073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.559316 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.559123 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.568032 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.568008 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks"] Jul 10 20:43:37.582714 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.582690 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:37.659799 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.659765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26qn\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-kube-api-access-v26qn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660005 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.659849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660005 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.659877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660005 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.659922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660169 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660169 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5d156fd-93b2-4e76-a510-e2e36422448d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660271 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660188 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660271 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660371 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660371 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660510 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660628 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660606 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.660877 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.660854 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.661133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.661118 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5d156fd-93b2-4e76-a510-e2e36422448d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.662796 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.662755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.663236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.663210 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.668329 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.668300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.668656 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.668633 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26qn\" (UniqueName: \"kubernetes.io/projected/e5d156fd-93b2-4e76-a510-e2e36422448d-kube-api-access-v26qn\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47djks\" (UID: \"e5d156fd-93b2-4e76-a510-e2e36422448d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.791834 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.791776 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:37.868960 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.868874 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:37.963677 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:37.963640 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:38.060325 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:38.060284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:38.945958 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:38.945934 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks"] Jul 10 20:43:38.948171 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:43:38.948147 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d156fd_93b2_4e76_a510_e2e36422448d.slice/crio-e3e337c6b4348c9f1a8fbe7628c92f64f0344dc2fa59ca5d777e73e3b639ada1 WatchSource:0}: Error finding container e3e337c6b4348c9f1a8fbe7628c92f64f0344dc2fa59ca5d777e73e3b639ada1: Status 404 returned error can't find the container with id e3e337c6b4348c9f1a8fbe7628c92f64f0344dc2fa59ca5d777e73e3b639ada1 Jul 10 20:43:39.588514 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.588485 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bkqfb" Jul 10 20:43:39.596268 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.596245 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-db5gc" Jul 10 20:43:39.651117 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.651078 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" event={"ID":"7f6a6fdc-3484-4755-9ac9-f01810bcd773","Type":"ContainerStarted","Data":"5ff7a1e67b1f907282b439b3731ae7745235f8aaabd4028ae88021fad3e2c876"} Jul 10 20:43:39.651298 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.651146 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:43:39.652461 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.652435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" event={"ID":"e5d156fd-93b2-4e76-a510-e2e36422448d","Type":"ContainerStarted","Data":"e3e337c6b4348c9f1a8fbe7628c92f64f0344dc2fa59ca5d777e73e3b639ada1"} Jul 10 20:43:39.654101 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.654079 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" event={"ID":"bdac9a85-5cf2-427b-878f-da31494bcb49","Type":"ContainerStarted","Data":"1ed1dd833d156324b590929556f4f2a8a0d3ef0c1a6a1b0132c43c41f0aa8a92"} Jul 10 20:43:39.654235 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.654219 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:43:39.666326 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.666268 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" podStartSLOduration=2.558246682 podStartE2EDuration="5.666249495s" totalImagesPullingTime="3.108002813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:39.665197702 +0000 UTC m=+528.549029047" watchObservedRunningTime="2026-07-10 20:43:39.666249495 +0000 UTC m=+528.550080840" Jul 10 20:43:39.683275 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:39.683229 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" podStartSLOduration=2.605270537 podStartE2EDuration="5.683214893s" totalImagesPullingTime="3.077944356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:39.680554026 +0000 UTC m=+528.564385369" watchObservedRunningTime="2026-07-10 20:43:39.683214893 +0000 UTC m=+528.567046236" Jul 10 20:43:49.171840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.171786 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:43:49.172130 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.171879 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:43:49.172130 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.171909 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:43:49.709144 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.709102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" event={"ID":"e5d156fd-93b2-4e76-a510-e2e36422448d","Type":"ContainerStarted","Data":"0d8609868f9e2354a19f792e8f6c20298f6514572320fbb5222bb1cfb852cf5d"} Jul 10 20:43:49.730690 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.730639 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" podStartSLOduration=2.509369983 podStartE2EDuration="12.730623633s" totalImagesPullingTime="10.22125365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:49.728392919 +0000 UTC m=+538.612224332" watchObservedRunningTime="2026-07-10 20:43:49.730623633 +0000 UTC m=+538.614454976" Jul 10 20:43:49.869581 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:49.869549 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:50.875934 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:50.875906 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:51.700658 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:51.700622 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:43:51.700785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:51.700704 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:43:51.718950 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:51.718918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:51.720046 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:51.720029 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47djks" Jul 10 20:43:52.725016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.724976 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" event={"ID":"36ca0227-498f-448a-a7ee-a27c55338904","Type":"ContainerStarted","Data":"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214"} Jul 10 20:43:52.725016 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.725009 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" podUID="36ca0227-498f-448a-a7ee-a27c55338904" containerName="discovery" containerID="cri-o://13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214" gracePeriod=30 Jul 10 20:43:52.725558 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.725343 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:52.727009 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.726991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:52.745114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.745072 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" podStartSLOduration=1.616688023 podStartE2EDuration="22.745057836s" totalImagesPullingTime="21.128369813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:43:52.743397508 +0000 UTC m=+541.627228853" watchObservedRunningTime="2026-07-10 20:43:52.745057836 +0000 UTC m=+541.628889178" Jul 10 20:43:52.969513 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:52.969489 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:53.105741 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.105702 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.105971 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.105758 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.105971 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.105849 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.105971 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.105906 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.106139 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.105973 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9f79\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.106139 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.106024 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.106139 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.106068 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts\") pod \"36ca0227-498f-448a-a7ee-a27c55338904\" (UID: \"36ca0227-498f-448a-a7ee-a27c55338904\") " Jul 10 20:43:53.106619 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.106579 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 20:43:53.108242 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108216 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token" (OuterVolumeSpecName: "istio-token") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:43:53.108242 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108213 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:43:53.108393 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108334 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs" (OuterVolumeSpecName: "local-certs") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:43:53.108474 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108459 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79" (OuterVolumeSpecName: "kube-api-access-t9f79") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "kube-api-access-t9f79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:43:53.108767 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108743 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts" (OuterVolumeSpecName: "cacerts") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:43:53.108842 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.108753 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "36ca0227-498f-448a-a7ee-a27c55338904" (UID: "36ca0227-498f-448a-a7ee-a27c55338904"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 20:43:53.207114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207080 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-dns-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207111 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9f79\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-kube-api-access-t9f79\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207114 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207121 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ca0227-498f-448a-a7ee-a27c55338904-istio-csr-ca-configmap\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207349 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207131 2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-cacerts\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207349 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207142 2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ca0227-498f-448a-a7ee-a27c55338904-local-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207349 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207152 2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ca0227-498f-448a-a7ee-a27c55338904-istio-kubeconfig\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.207349 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.207160 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ca0227-498f-448a-a7ee-a27c55338904-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:43:53.730728 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.730697 2566 generic.go:356] "Generic (PLEG): container finished" podID="36ca0227-498f-448a-a7ee-a27c55338904" containerID="13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214" exitCode=0 Jul 10 20:43:53.731249 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.730747 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" Jul 10 20:43:53.731249 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.730776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" event={"ID":"36ca0227-498f-448a-a7ee-a27c55338904","Type":"ContainerDied","Data":"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214"} Jul 10 20:43:53.731249 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.730858 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs" event={"ID":"36ca0227-498f-448a-a7ee-a27c55338904","Type":"ContainerDied","Data":"812e86621e67ab4c94dde99c917847c3d5b85bc5d12b2a6d39887b54520aba84"} Jul 10 20:43:53.731249 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.730892 2566 scope.go:117] "RemoveContainer" containerID="13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214" Jul 10 20:43:53.739669 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.739654 2566 scope.go:117] "RemoveContainer" containerID="13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214" Jul 10 20:43:53.739942 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:43:53.739924 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214\": container with ID starting with 13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214 not found: ID does not exist" containerID="13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214" Jul 10 20:43:53.739994 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.739951 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214"} err="failed to get container status \"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214\": rpc error: code = NotFound desc = could not find container \"13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214\": container with ID starting with 13eaf1cbc76c10c810a575f222b8d33fda1baae100c82fc6c3ba69f173f9d214 not found: ID does not exist" Jul 10 20:43:53.748463 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.748439 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:53.750218 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:53.750198 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pzmcs"] Jul 10 20:43:55.596976 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:55.596943 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36ca0227-498f-448a-a7ee-a27c55338904" path="/var/lib/kubelet/pods/36ca0227-498f-448a-a7ee-a27c55338904/volumes" Jul 10 20:43:57.178073 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:57.178041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:43:57.281108 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:43:57.281080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:10.664907 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:10.664868 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4rtrm" Jul 10 20:44:10.668077 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:10.668055 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4dpkm" Jul 10 20:44:24.674777 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.674733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:24.772234 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.772190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:24.792559 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.792530 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:24.793067 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.793052 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36ca0227-498f-448a-a7ee-a27c55338904" containerName="discovery" Jul 10 20:44:24.793115 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.793069 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36ca0227-498f-448a-a7ee-a27c55338904" containerName="discovery" Jul 10 20:44:24.793152 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.793138 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36ca0227-498f-448a-a7ee-a27c55338904" containerName="discovery" Jul 10 20:44:24.799942 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.799916 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:24.802474 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.802452 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 20:44:24.802474 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.802460 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 20:44:24.803500 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.803480 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-99p7l\"" Jul 10 20:44:24.806436 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.806415 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:24.846530 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.846501 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:24.893604 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.893574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qts5\" (UniqueName: \"kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5\") pod \"kuadrant-operator-catalog-7dp8v\" (UID: \"b95388a5-e2e9-42f9-9b33-7f05f44c2b37\") " pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:24.994956 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:24.994925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qts5\" (UniqueName: \"kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5\") pod \"kuadrant-operator-catalog-7dp8v\" (UID: \"b95388a5-e2e9-42f9-9b33-7f05f44c2b37\") " pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:25.006302 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.006278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qts5\" (UniqueName: \"kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5\") pod \"kuadrant-operator-catalog-7dp8v\" (UID: \"b95388a5-e2e9-42f9-9b33-7f05f44c2b37\") " pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:25.112048 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.112014 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:25.163438 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.163353 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:25.244733 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.244711 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:25.247004 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:44:25.246944 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95388a5_e2e9_42f9_9b33_7f05f44c2b37.slice/crio-f9dd8f5d4b0e5c6798ed49d5fd59aa7ac487f951448aef2d3233f09f156a3fab WatchSource:0}: Error finding container f9dd8f5d4b0e5c6798ed49d5fd59aa7ac487f951448aef2d3233f09f156a3fab: Status 404 returned error can't find the container with id f9dd8f5d4b0e5c6798ed49d5fd59aa7ac487f951448aef2d3233f09f156a3fab Jul 10 20:44:25.374033 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.373999 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9zwg5"] Jul 10 20:44:25.377090 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.377074 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:25.385631 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.385609 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9zwg5"] Jul 10 20:44:25.399179 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.399159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pllkx\" (UniqueName: \"kubernetes.io/projected/19d1a82a-4c4f-405d-bca4-e97df73530e7-kube-api-access-pllkx\") pod \"kuadrant-operator-catalog-9zwg5\" (UID: \"19d1a82a-4c4f-405d-bca4-e97df73530e7\") " pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:25.500333 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.500234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pllkx\" (UniqueName: \"kubernetes.io/projected/19d1a82a-4c4f-405d-bca4-e97df73530e7-kube-api-access-pllkx\") pod \"kuadrant-operator-catalog-9zwg5\" (UID: \"19d1a82a-4c4f-405d-bca4-e97df73530e7\") " pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:25.511524 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.511498 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pllkx\" (UniqueName: \"kubernetes.io/projected/19d1a82a-4c4f-405d-bca4-e97df73530e7-kube-api-access-pllkx\") pod \"kuadrant-operator-catalog-9zwg5\" (UID: \"19d1a82a-4c4f-405d-bca4-e97df73530e7\") " pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:25.598226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.598194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:25.687697 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.687661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:25.806800 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.806472 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:25.837502 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.837480 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9zwg5"] Jul 10 20:44:25.861389 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:44:25.861359 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d1a82a_4c4f_405d_bca4_e97df73530e7.slice/crio-b866b7a67e39f2ead052bc1c55c8b45da8d169e0125cb4a056f95675c1fdfc63 WatchSource:0}: Error finding container b866b7a67e39f2ead052bc1c55c8b45da8d169e0125cb4a056f95675c1fdfc63: Status 404 returned error can't find the container with id b866b7a67e39f2ead052bc1c55c8b45da8d169e0125cb4a056f95675c1fdfc63 Jul 10 20:44:25.888563 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.888524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" event={"ID":"b95388a5-e2e9-42f9-9b33-7f05f44c2b37","Type":"ContainerStarted","Data":"f9dd8f5d4b0e5c6798ed49d5fd59aa7ac487f951448aef2d3233f09f156a3fab"} Jul 10 20:44:25.889955 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:25.889926 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" event={"ID":"19d1a82a-4c4f-405d-bca4-e97df73530e7","Type":"ContainerStarted","Data":"b866b7a67e39f2ead052bc1c55c8b45da8d169e0125cb4a056f95675c1fdfc63"} Jul 10 20:44:27.902710 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:27.902677 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" event={"ID":"b95388a5-e2e9-42f9-9b33-7f05f44c2b37","Type":"ContainerStarted","Data":"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8"} Jul 10 20:44:27.903201 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:27.902748 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" podUID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" containerName="registry-server" containerID="cri-o://914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8" gracePeriod=2 Jul 10 20:44:27.904312 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:27.904287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" event={"ID":"19d1a82a-4c4f-405d-bca4-e97df73530e7","Type":"ContainerStarted","Data":"0242391b60c3ca72810626cc4cbd46c298cfa7d3ad91a1749b0f449c2e6ea92d"} Jul 10 20:44:27.931195 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:27.931151 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" podStartSLOduration=1.958330149 podStartE2EDuration="3.931135757s" totalImagesPullingTime="1.972805608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:27.930573579 +0000 UTC m=+576.814404923" watchObservedRunningTime="2026-07-10 20:44:27.931135757 +0000 UTC m=+576.814967100" Jul 10 20:44:27.956176 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:27.956129 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" podStartSLOduration=1.593902834 podStartE2EDuration="2.956113088s" totalImagesPullingTime="1.362210254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:27.95602278 +0000 UTC m=+576.839854209" watchObservedRunningTime="2026-07-10 20:44:27.956113088 +0000 UTC m=+576.839944430" Jul 10 20:44:28.166887 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.166824 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:28.227206 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.227174 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qts5\" (UniqueName: \"kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5\") pod \"b95388a5-e2e9-42f9-9b33-7f05f44c2b37\" (UID: \"b95388a5-e2e9-42f9-9b33-7f05f44c2b37\") " Jul 10 20:44:28.229315 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.229290 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5" (OuterVolumeSpecName: "kube-api-access-9qts5") pod "b95388a5-e2e9-42f9-9b33-7f05f44c2b37" (UID: "b95388a5-e2e9-42f9-9b33-7f05f44c2b37"). InnerVolumeSpecName "kube-api-access-9qts5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:44:28.327979 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.327939 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qts5\" (UniqueName: \"kubernetes.io/projected/b95388a5-e2e9-42f9-9b33-7f05f44c2b37-kube-api-access-9qts5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:44:28.909827 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.909772 2566 generic.go:356] "Generic (PLEG): container finished" podID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" containerID="914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8" exitCode=0 Jul 10 20:44:28.910236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.909841 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" Jul 10 20:44:28.910236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.909859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" event={"ID":"b95388a5-e2e9-42f9-9b33-7f05f44c2b37","Type":"ContainerDied","Data":"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8"} Jul 10 20:44:28.910236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.909910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7dp8v" event={"ID":"b95388a5-e2e9-42f9-9b33-7f05f44c2b37","Type":"ContainerDied","Data":"f9dd8f5d4b0e5c6798ed49d5fd59aa7ac487f951448aef2d3233f09f156a3fab"} Jul 10 20:44:28.910236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.909931 2566 scope.go:117] "RemoveContainer" containerID="914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8" Jul 10 20:44:28.919150 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.919135 2566 scope.go:117] "RemoveContainer" containerID="914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8" Jul 10 20:44:28.919399 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:44:28.919376 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8\": container with ID starting with 914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8 not found: ID does not exist" containerID="914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8" Jul 10 20:44:28.919467 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.919414 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8"} err="failed to get container status \"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8\": rpc error: code = NotFound desc = could not find container \"914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8\": container with ID starting with 914aa066c69a5ffd885408ba62f00360338b5c7d91ef64680b92770338a53ad8 not found: ID does not exist" Jul 10 20:44:28.930218 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.930194 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:28.933565 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:28.933542 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7dp8v"] Jul 10 20:44:29.596311 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:29.596280 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" path="/var/lib/kubelet/pods/b95388a5-e2e9-42f9-9b33-7f05f44c2b37/volumes" Jul 10 20:44:35.688103 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:35.688059 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:35.688617 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:35.688139 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:35.710896 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:35.710867 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:35.973306 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:35.973229 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-9zwg5" Jul 10 20:44:37.492465 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:37.492402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:44:51.514989 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:51.514968 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:44:51.515438 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:51.515421 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:44:51.519138 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:51.519122 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:44:51.519615 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:44:51.519595 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:45:00.324052 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:00.324015 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:02.974626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.974593 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww"] Jul 10 20:45:02.975040 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.975027 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" containerName="registry-server" Jul 10 20:45:02.975040 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.975041 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" containerName="registry-server" Jul 10 20:45:02.975162 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.975151 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b95388a5-e2e9-42f9-9b33-7f05f44c2b37" containerName="registry-server" Jul 10 20:45:02.978754 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.978737 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:02.982105 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.982086 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7zfnv\"" Jul 10 20:45:02.990530 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:02.990502 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww"] Jul 10 20:45:03.053994 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.053957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghpgc\" (UniqueName: \"kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc\") pod \"limitador-operator-controller-manager-859d7b78bd-xtcww\" (UID: \"488c2808-3496-4e5d-b058-bb034bc43ae6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:03.155037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.155001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghpgc\" (UniqueName: \"kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc\") pod \"limitador-operator-controller-manager-859d7b78bd-xtcww\" (UID: \"488c2808-3496-4e5d-b058-bb034bc43ae6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:03.171684 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.171654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghpgc\" (UniqueName: \"kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc\") pod \"limitador-operator-controller-manager-859d7b78bd-xtcww\" (UID: \"488c2808-3496-4e5d-b058-bb034bc43ae6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:03.289415 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.289376 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:03.426089 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.426064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww"] Jul 10 20:45:03.428467 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:03.428436 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488c2808_3496_4e5d_b058_bb034bc43ae6.slice/crio-33cbc1bfaee6ebe634e26768d28b312b83806f704f47ec72c2f37d883f3df43e WatchSource:0}: Error finding container 33cbc1bfaee6ebe634e26768d28b312b83806f704f47ec72c2f37d883f3df43e: Status 404 returned error can't find the container with id 33cbc1bfaee6ebe634e26768d28b312b83806f704f47ec72c2f37d883f3df43e Jul 10 20:45:03.924093 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:03.924056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:04.089379 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:04.089340 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" event={"ID":"488c2808-3496-4e5d-b058-bb034bc43ae6","Type":"ContainerStarted","Data":"33cbc1bfaee6ebe634e26768d28b312b83806f704f47ec72c2f37d883f3df43e"} Jul 10 20:45:05.996306 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:05.996275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:06.101362 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:06.101313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:06.101555 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:06.101418 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" event={"ID":"488c2808-3496-4e5d-b058-bb034bc43ae6","Type":"ContainerStarted","Data":"309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029"} Jul 10 20:45:06.101608 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:06.101583 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:06.120769 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:06.120713 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" podStartSLOduration=2.272608153 podStartE2EDuration="4.120698305s" totalImagesPullingTime="1.848090152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:06.119337044 +0000 UTC m=+615.003168379" watchObservedRunningTime="2026-07-10 20:45:06.120698305 +0000 UTC m=+615.004529647" Jul 10 20:45:08.137918 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.137883 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv"] Jul 10 20:45:08.141639 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.141615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.146665 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.146648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-94hz6\"" Jul 10 20:45:08.155783 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.155761 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv"] Jul 10 20:45:08.194899 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.194861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xqb\" (UniqueName: \"kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.195050 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.194957 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.295497 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.295461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.295662 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.295539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xqb\" (UniqueName: \"kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.295880 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.295860 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.305272 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.305248 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xqb\" (UniqueName: \"kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.451548 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.451461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:08.587703 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:08.587679 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv"] Jul 10 20:45:08.590001 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:08.589974 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2523f464_ac21_4d11_a24e_7a9d14e88b07.slice/crio-5c5654c0f338ed6c086a2a838b11b63ae9a4c4f6ed91abbbc429de6c7ad37125 WatchSource:0}: Error finding container 5c5654c0f338ed6c086a2a838b11b63ae9a4c4f6ed91abbbc429de6c7ad37125: Status 404 returned error can't find the container with id 5c5654c0f338ed6c086a2a838b11b63ae9a4c4f6ed91abbbc429de6c7ad37125 Jul 10 20:45:09.118102 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:09.118070 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" event={"ID":"2523f464-ac21-4d11-a24e-7a9d14e88b07","Type":"ContainerStarted","Data":"5c5654c0f338ed6c086a2a838b11b63ae9a4c4f6ed91abbbc429de6c7ad37125"} Jul 10 20:45:09.155311 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:09.155271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:09.296396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:09.296362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:10.367406 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:10.367356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:11.723377 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.723343 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n"] Jul 10 20:45:11.727270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.727245 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:11.737586 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.737563 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 20:45:11.737707 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.737565 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ct29v\"" Jul 10 20:45:11.752291 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.752264 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n"] Jul 10 20:45:11.825582 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.825541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4sfw\" (UniqueName: \"kubernetes.io/projected/75a98ad8-e691-4d5a-83ce-1e9bb26da412-kube-api-access-p4sfw\") pod \"dns-operator-controller-manager-864dd78c4c-m5w5n\" (UID: \"75a98ad8-e691-4d5a-83ce-1e9bb26da412\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:11.926790 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.926756 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4sfw\" (UniqueName: \"kubernetes.io/projected/75a98ad8-e691-4d5a-83ce-1e9bb26da412-kube-api-access-p4sfw\") pod \"dns-operator-controller-manager-864dd78c4c-m5w5n\" (UID: \"75a98ad8-e691-4d5a-83ce-1e9bb26da412\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:11.946696 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:11.946668 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4sfw\" (UniqueName: \"kubernetes.io/projected/75a98ad8-e691-4d5a-83ce-1e9bb26da412-kube-api-access-p4sfw\") pod \"dns-operator-controller-manager-864dd78c4c-m5w5n\" (UID: \"75a98ad8-e691-4d5a-83ce-1e9bb26da412\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:12.012794 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:12.012751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:12.040981 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:12.040951 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:12.139376 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:12.139329 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:12.199306 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:12.199283 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n"] Jul 10 20:45:12.201993 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:12.201958 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a98ad8_e691_4d5a_83ce_1e9bb26da412.slice/crio-759caa8ddf144d8590e96966a8094b3d15c031f55fea8d7592dac296a20328a6 WatchSource:0}: Error finding container 759caa8ddf144d8590e96966a8094b3d15c031f55fea8d7592dac296a20328a6: Status 404 returned error can't find the container with id 759caa8ddf144d8590e96966a8094b3d15c031f55fea8d7592dac296a20328a6 Jul 10 20:45:12.785006 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:12.784977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:13.145799 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:13.145684 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" event={"ID":"75a98ad8-e691-4d5a-83ce-1e9bb26da412","Type":"ContainerStarted","Data":"759caa8ddf144d8590e96966a8094b3d15c031f55fea8d7592dac296a20328a6"} Jul 10 20:45:14.498794 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:14.498749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:14.786173 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:14.786084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:15.953396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:15.953366 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx"] Jul 10 20:45:15.956957 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:15.956937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:15.968767 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:15.968743 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-q8kn7\"" Jul 10 20:45:15.987626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:15.987594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx"] Jul 10 20:45:16.067935 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.067892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4772v\" (UniqueName: \"kubernetes.io/projected/7a3edc5d-f465-442a-8858-b2b76500c072-kube-api-access-4772v\") pod \"authorino-operator-75cc8d5cb7-lzwzx\" (UID: \"7a3edc5d-f465-442a-8858-b2b76500c072\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:16.163943 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.163904 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" event={"ID":"2523f464-ac21-4d11-a24e-7a9d14e88b07","Type":"ContainerStarted","Data":"77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e"} Jul 10 20:45:16.164171 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.164023 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:16.166356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.166327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" event={"ID":"75a98ad8-e691-4d5a-83ce-1e9bb26da412","Type":"ContainerStarted","Data":"1cb3c1734e931206f030795d202caee213630c3b47cde2cc01bffadd68e384a5"} Jul 10 20:45:16.166561 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.166536 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:16.168942 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.168918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4772v\" (UniqueName: \"kubernetes.io/projected/7a3edc5d-f465-442a-8858-b2b76500c072-kube-api-access-4772v\") pod \"authorino-operator-75cc8d5cb7-lzwzx\" (UID: \"7a3edc5d-f465-442a-8858-b2b76500c072\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:16.184850 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.184800 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4772v\" (UniqueName: \"kubernetes.io/projected/7a3edc5d-f465-442a-8858-b2b76500c072-kube-api-access-4772v\") pod \"authorino-operator-75cc8d5cb7-lzwzx\" (UID: \"7a3edc5d-f465-442a-8858-b2b76500c072\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:16.268773 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.268733 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:16.314780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.314714 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" podStartSLOduration=2.330796612 podStartE2EDuration="5.314688541s" totalImagesPullingTime="2.983891929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:16.312590989 +0000 UTC m=+625.196422371" watchObservedRunningTime="2026-07-10 20:45:16.314688541 +0000 UTC m=+625.198519888" Jul 10 20:45:16.316735 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.316682 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" podStartSLOduration=1.724918811 podStartE2EDuration="8.31666761s" totalImagesPullingTime="6.591748799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:16.217601814 +0000 UTC m=+625.101433157" watchObservedRunningTime="2026-07-10 20:45:16.31666761 +0000 UTC m=+625.200498954" Jul 10 20:45:16.443522 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:16.443482 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx"] Jul 10 20:45:16.518941 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:16.518882 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3edc5d_f465_442a_8858_b2b76500c072.slice/crio-47ead0c0a9742df16c0a81cd8b2fd5250c55fce9950756d448e37daac679c518 WatchSource:0}: Error finding container 47ead0c0a9742df16c0a81cd8b2fd5250c55fce9950756d448e37daac679c518: Status 404 returned error can't find the container with id 47ead0c0a9742df16c0a81cd8b2fd5250c55fce9950756d448e37daac679c518 Jul 10 20:45:17.078721 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:17.078683 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:17.110205 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:17.110181 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:17.172783 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:17.172750 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" event={"ID":"7a3edc5d-f465-442a-8858-b2b76500c072","Type":"ContainerStarted","Data":"47ead0c0a9742df16c0a81cd8b2fd5250c55fce9950756d448e37daac679c518"} Jul 10 20:45:17.369110 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:17.369016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:18.618785 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.618748 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22"] Jul 10 20:45:18.622681 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.622659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.636664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.636643 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 10 20:45:18.636851 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.636688 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 10 20:45:18.636921 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.636909 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vbv2p\"" Jul 10 20:45:18.657694 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.657663 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22"] Jul 10 20:45:18.675710 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.675662 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:18.694735 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.694707 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a798c131-c3bc-41f4-b787-619b32e3335a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.694912 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.694759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj972\" (UniqueName: \"kubernetes.io/projected/a798c131-c3bc-41f4-b787-619b32e3335a-kube-api-access-dj972\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.694912 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.694802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.795556 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.795520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a798c131-c3bc-41f4-b787-619b32e3335a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.795746 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.795573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj972\" (UniqueName: \"kubernetes.io/projected/a798c131-c3bc-41f4-b787-619b32e3335a-kube-api-access-dj972\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.795746 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.795599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.795878 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:18.795766 2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 10 20:45:18.795878 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:18.795860 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert podName:a798c131-c3bc-41f4-b787-619b32e3335a nodeName:}" failed. No retries permitted until 2026-07-10 20:45:19.295841185 +0000 UTC m=+628.179672506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-d7m22" (UID: "a798c131-c3bc-41f4-b787-619b32e3335a") : secret "plugin-serving-cert" not found Jul 10 20:45:18.796311 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.796289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a798c131-c3bc-41f4-b787-619b32e3335a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:18.826852 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:18.826792 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj972\" (UniqueName: \"kubernetes.io/projected/a798c131-c3bc-41f4-b787-619b32e3335a-kube-api-access-dj972\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:19.022758 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.022735 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:19.185260 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.185227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" event={"ID":"7a3edc5d-f465-442a-8858-b2b76500c072","Type":"ContainerStarted","Data":"1f8aef961fec3b1b64cf705b89a5e4d17d5246a926f018d87ad907db6fcff496"} Jul 10 20:45:19.185438 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.185320 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:19.242545 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.242496 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" podStartSLOduration=1.811346586 podStartE2EDuration="4.242467545s" totalImagesPullingTime="2.431120959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:19.234573631 +0000 UTC m=+628.118404985" watchObservedRunningTime="2026-07-10 20:45:19.242467545 +0000 UTC m=+628.126298887" Jul 10 20:45:19.300607 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.300569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:19.302939 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.302912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a798c131-c3bc-41f4-b787-619b32e3335a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-d7m22\" (UID: \"a798c131-c3bc-41f4-b787-619b32e3335a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:19.536866 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.536785 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" Jul 10 20:45:19.747333 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:19.747311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22"] Jul 10 20:45:19.748980 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:19.748955 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda798c131_c3bc_41f4_b787_619b32e3335a.slice/crio-990bce13dc51285e9767b7e5c3ae61d690d4526c2a30ab4c6ec351a011ae3952 WatchSource:0}: Error finding container 990bce13dc51285e9767b7e5c3ae61d690d4526c2a30ab4c6ec351a011ae3952: Status 404 returned error can't find the container with id 990bce13dc51285e9767b7e5c3ae61d690d4526c2a30ab4c6ec351a011ae3952 Jul 10 20:45:20.191271 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:20.191237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" event={"ID":"a798c131-c3bc-41f4-b787-619b32e3335a","Type":"ContainerStarted","Data":"990bce13dc51285e9767b7e5c3ae61d690d4526c2a30ab4c6ec351a011ae3952"} Jul 10 20:45:20.689683 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:20.689646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:21.086999 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:21.086960 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:27.130345 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:27.130292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:27.175235 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:27.175206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:27.175409 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:27.175396 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-m5w5n" Jul 10 20:45:27.420732 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:27.420645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:29.227451 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.227419 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv"] Jul 10 20:45:29.227911 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.227702 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" containerName="manager" containerID="cri-o://77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e" gracePeriod=2 Jul 10 20:45:29.267372 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.267342 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv"] Jul 10 20:45:29.283721 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.283677 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww"] Jul 10 20:45:29.284006 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.283962 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" containerName="manager" containerID="cri-o://309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029" gracePeriod=2 Jul 10 20:45:29.286359 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:29.286324 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:29.354192 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.354099 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww"] Jul 10 20:45:29.356499 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.356467 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:29.357065 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357043 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" containerName="manager" Jul 10 20:45:29.357065 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357064 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" containerName="manager" Jul 10 20:45:29.357248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357099 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" containerName="manager" Jul 10 20:45:29.357248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357108 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" containerName="manager" Jul 10 20:45:29.357248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357204 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" containerName="manager" Jul 10 20:45:29.357248 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.357221 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" containerName="manager" Jul 10 20:45:29.361784 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.361760 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.364429 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:29.364305 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:29.368413 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:29.368261 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:29.382725 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.382693 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf"] Jul 10 20:45:29.387894 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.387871 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:29.467309 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.467271 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:29.469547 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.469520 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf"] Jul 10 20:45:29.502740 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.502712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.502952 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.502902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62h64\" (UniqueName: \"kubernetes.io/projected/7d0121b1-9b69-43fb-9c60-f7e5e671936b-kube-api-access-62h64\") pod \"limitador-operator-controller-manager-859d7b78bd-57gpf\" (UID: \"7d0121b1-9b69-43fb-9c60-f7e5e671936b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:29.502952 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.502946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhks\" (UniqueName: \"kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.549162 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:29.549116 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:29.556327 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:29.556289 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:29.604275 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.604240 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62h64\" (UniqueName: \"kubernetes.io/projected/7d0121b1-9b69-43fb-9c60-f7e5e671936b-kube-api-access-62h64\") pod \"limitador-operator-controller-manager-859d7b78bd-57gpf\" (UID: \"7d0121b1-9b69-43fb-9c60-f7e5e671936b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:29.604275 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.604280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhks\" (UniqueName: \"kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.604561 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.604352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.604753 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.604732 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.631537 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.631507 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhks\" (UniqueName: \"kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zxd4k\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.632532 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.632501 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62h64\" (UniqueName: \"kubernetes.io/projected/7d0121b1-9b69-43fb-9c60-f7e5e671936b-kube-api-access-62h64\") pod \"limitador-operator-controller-manager-859d7b78bd-57gpf\" (UID: \"7d0121b1-9b69-43fb-9c60-f7e5e671936b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:29.676408 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.676371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:29.702014 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:29.701978 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:30.194490 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:30.194461 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lzwzx" Jul 10 20:45:32.649270 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.649233 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:45:32.707730 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.707684 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:45:32.707936 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.707835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.839289 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.839248 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.839432 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.839371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78pt\" (UniqueName: \"kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.940725 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.940642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78pt\" (UniqueName: \"kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.940916 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.940742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.941181 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.941155 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:32.957664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:32.957631 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78pt\" (UniqueName: \"kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt\") pod \"kuadrant-operator-controller-manager-86878c769f-tlkl7\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:33.021404 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:33.021362 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:33.444697 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:33.444648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:33.695661 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:33.695583 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:33.815070 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:33.815034 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:45:42.919162 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:42.918862 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:42.921590 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:42.921515 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:42.924000 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:42.923947 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd668e025_57b7_4109_8fe3_0823a0156319.slice/crio-73650910eae1def1cde80cfe4e435bfe258835808bf6ce34de078e19f5b10f25 WatchSource:0}: Error finding container 73650910eae1def1cde80cfe4e435bfe258835808bf6ce34de078e19f5b10f25: Status 404 returned error can't find the container with id 73650910eae1def1cde80cfe4e435bfe258835808bf6ce34de078e19f5b10f25 Jul 10 20:45:42.924183 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:42.924086 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:45:42.924579 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:42.924504 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:42.927293 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:42.927166 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:42.935022 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:42.934987 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:42.960572 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:42.960550 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:42.970089 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:42.970063 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod030973c9_12f6_4ee6_867b_f0015ade00de.slice/crio-bf0973cb96b3e3b2b5bae397a97af3311706d644f3f41a2d7d8d84bee61c4c1c WatchSource:0}: Error finding container bf0973cb96b3e3b2b5bae397a97af3311706d644f3f41a2d7d8d84bee61c4c1c: Status 404 returned error can't find the container with id bf0973cb96b3e3b2b5bae397a97af3311706d644f3f41a2d7d8d84bee61c4c1c Jul 10 20:45:42.979132 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:42.979101 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf"] Jul 10 20:45:42.982987 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:45:42.982965 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0121b1_9b69_43fb_9c60_f7e5e671936b.slice/crio-e11d72813ccdce248a683205bd3af2d0d742424b662136a69a680f369c9eebce WatchSource:0}: Error finding container e11d72813ccdce248a683205bd3af2d0d742424b662136a69a680f369c9eebce: Status 404 returned error can't find the container with id e11d72813ccdce248a683205bd3af2d0d742424b662136a69a680f369c9eebce Jul 10 20:45:43.043202 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.043174 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghpgc\" (UniqueName: \"kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc\") pod \"488c2808-3496-4e5d-b058-bb034bc43ae6\" (UID: \"488c2808-3496-4e5d-b058-bb034bc43ae6\") " Jul 10 20:45:43.043321 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.043243 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5xqb\" (UniqueName: \"kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb\") pod \"2523f464-ac21-4d11-a24e-7a9d14e88b07\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " Jul 10 20:45:43.043378 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.043333 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume\") pod \"2523f464-ac21-4d11-a24e-7a9d14e88b07\" (UID: \"2523f464-ac21-4d11-a24e-7a9d14e88b07\") " Jul 10 20:45:43.049848 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.044011 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2523f464-ac21-4d11-a24e-7a9d14e88b07" (UID: "2523f464-ac21-4d11-a24e-7a9d14e88b07"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:45:43.049848 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.046183 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc" (OuterVolumeSpecName: "kube-api-access-ghpgc") pod "488c2808-3496-4e5d-b058-bb034bc43ae6" (UID: "488c2808-3496-4e5d-b058-bb034bc43ae6"). InnerVolumeSpecName "kube-api-access-ghpgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:45:43.049848 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.046202 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb" (OuterVolumeSpecName: "kube-api-access-x5xqb") pod "2523f464-ac21-4d11-a24e-7a9d14e88b07" (UID: "2523f464-ac21-4d11-a24e-7a9d14e88b07"). InnerVolumeSpecName "kube-api-access-x5xqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:45:43.144019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.143978 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2523f464-ac21-4d11-a24e-7a9d14e88b07-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:45:43.144019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.144006 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghpgc\" (UniqueName: \"kubernetes.io/projected/488c2808-3496-4e5d-b058-bb034bc43ae6-kube-api-access-ghpgc\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:45:43.144019 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.144015 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5xqb\" (UniqueName: \"kubernetes.io/projected/2523f464-ac21-4d11-a24e-7a9d14e88b07-kube-api-access-x5xqb\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:45:43.332457 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.332422 2566 generic.go:356] "Generic (PLEG): container finished" podID="2523f464-ac21-4d11-a24e-7a9d14e88b07" containerID="77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e" exitCode=0 Jul 10 20:45:43.332655 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.332478 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:43.332655 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.332524 2566 scope.go:117] "RemoveContainer" containerID="77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e" Jul 10 20:45:43.333949 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.333923 2566 generic.go:356] "Generic (PLEG): container finished" podID="488c2808-3496-4e5d-b058-bb034bc43ae6" containerID="309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029" exitCode=0 Jul 10 20:45:43.334095 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.333958 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:43.334984 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.334949 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:43.335977 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.335946 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" event={"ID":"030973c9-12f6-4ee6-867b-f0015ade00de","Type":"ContainerStarted","Data":"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb"} Jul 10 20:45:43.336096 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.336031 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" event={"ID":"030973c9-12f6-4ee6-867b-f0015ade00de","Type":"ContainerStarted","Data":"bf0973cb96b3e3b2b5bae397a97af3311706d644f3f41a2d7d8d84bee61c4c1c"} Jul 10 20:45:43.336096 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.336062 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:43.337980 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.337953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" event={"ID":"d668e025-57b7-4109-8fe3-0823a0156319","Type":"ContainerStarted","Data":"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806"} Jul 10 20:45:43.338083 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.338001 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" event={"ID":"d668e025-57b7-4109-8fe3-0823a0156319","Type":"ContainerStarted","Data":"73650910eae1def1cde80cfe4e435bfe258835808bf6ce34de078e19f5b10f25"} Jul 10 20:45:43.338214 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.338194 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:43.338490 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.338464 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:43.339773 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.339749 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" event={"ID":"a798c131-c3bc-41f4-b787-619b32e3335a","Type":"ContainerStarted","Data":"069facef8e5b5aa00c74d04bbfd792984ba8d739430186f15f19a4da6b2a7e15"} Jul 10 20:45:43.341776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.341529 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" event={"ID":"7d0121b1-9b69-43fb-9c60-f7e5e671936b","Type":"ContainerStarted","Data":"dad30b6d05339811d64be48c912375892aca03651b566778d97710a0e32b8880"} Jul 10 20:45:43.341776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.341573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" event={"ID":"7d0121b1-9b69-43fb-9c60-f7e5e671936b","Type":"ContainerStarted","Data":"e11d72813ccdce248a683205bd3af2d0d742424b662136a69a680f369c9eebce"} Jul 10 20:45:43.341776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.341596 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:43.343489 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.343478 2566 scope.go:117] "RemoveContainer" containerID="77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e" Jul 10 20:45:43.343749 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.343726 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e\": container with ID starting with 77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e not found: ID does not exist" containerID="77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e" Jul 10 20:45:43.343859 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.343759 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e"} err="failed to get container status \"77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e\": rpc error: code = NotFound desc = could not find container \"77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e\": container with ID starting with 77f00128b14f763f622336f639fbf5517b43c37a650d682460591e75abe99a1e not found: ID does not exist" Jul 10 20:45:43.343859 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.343782 2566 scope.go:117] "RemoveContainer" containerID="309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029" Jul 10 20:45:43.351634 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.351614 2566 scope.go:117] "RemoveContainer" containerID="309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029" Jul 10 20:45:43.351906 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.351887 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029\": container with ID starting with 309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029 not found: ID does not exist" containerID="309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029" Jul 10 20:45:43.351982 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.351910 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029"} err="failed to get container status \"309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029\": rpc error: code = NotFound desc = could not find container \"309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029\": container with ID starting with 309a022c6ce2b2857af0defbe0ef72aac81154df9a2ba10fadf4e2584c052029 not found: ID does not exist" Jul 10 20:45:43.363823 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.363762 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" podStartSLOduration=11.363749216 podStartE2EDuration="11.363749216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.362765723 +0000 UTC m=+652.246597109" watchObservedRunningTime="2026-07-10 20:45:43.363749216 +0000 UTC m=+652.247580558" Jul 10 20:45:43.366182 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.366043 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:43.368025 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.367965 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:43.412587 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.412536 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" podStartSLOduration=14.412517235 podStartE2EDuration="14.412517235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.40932567 +0000 UTC m=+652.293157013" watchObservedRunningTime="2026-07-10 20:45:43.412517235 +0000 UTC m=+652.296348577" Jul 10 20:45:43.412927 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.412894 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" podStartSLOduration=14.412884891 podStartE2EDuration="14.412884891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.391372877 +0000 UTC m=+652.275204214" watchObservedRunningTime="2026-07-10 20:45:43.412884891 +0000 UTC m=+652.296716234" Jul 10 20:45:43.431206 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.431153 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-d7m22" podStartSLOduration=2.359128981 podStartE2EDuration="25.431132859s" totalImagesPullingTime="23.072003878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:45:43.429926557 +0000 UTC m=+652.313757901" watchObservedRunningTime="2026-07-10 20:45:43.431132859 +0000 UTC m=+652.314964203" Jul 10 20:45:43.434269 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.434235 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xtcww\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xtcww" Jul 10 20:45:43.435980 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:43.435953 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m2qnv\" is forbidden: User \"system:node:ip-10-0-137-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-185.ec2.internal' and this object" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m2qnv" Jul 10 20:45:43.597210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.597122 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2523f464-ac21-4d11-a24e-7a9d14e88b07" path="/var/lib/kubelet/pods/2523f464-ac21-4d11-a24e-7a9d14e88b07/volumes" Jul 10 20:45:43.597464 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:43.597452 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="488c2808-3496-4e5d-b058-bb034bc43ae6" path="/var/lib/kubelet/pods/488c2808-3496-4e5d-b058-bb034bc43ae6/volumes" Jul 10 20:45:54.350539 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.350506 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:45:54.350983 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.350592 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:54.350983 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.350774 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-57gpf" Jul 10 20:45:54.474524 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.474493 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:54.474711 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.474667 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" podUID="030973c9-12f6-4ee6-867b-f0015ade00de" containerName="manager" containerID="cri-o://257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb" gracePeriod=10 Jul 10 20:45:54.735170 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.735150 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:54.848458 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.848427 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmhks\" (UniqueName: \"kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks\") pod \"030973c9-12f6-4ee6-867b-f0015ade00de\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " Jul 10 20:45:54.848611 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.848469 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume\") pod \"030973c9-12f6-4ee6-867b-f0015ade00de\" (UID: \"030973c9-12f6-4ee6-867b-f0015ade00de\") " Jul 10 20:45:54.848923 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.848881 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "030973c9-12f6-4ee6-867b-f0015ade00de" (UID: "030973c9-12f6-4ee6-867b-f0015ade00de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:45:54.850437 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.850412 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks" (OuterVolumeSpecName: "kube-api-access-pmhks") pod "030973c9-12f6-4ee6-867b-f0015ade00de" (UID: "030973c9-12f6-4ee6-867b-f0015ade00de"). InnerVolumeSpecName "kube-api-access-pmhks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:45:54.949452 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.949382 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/030973c9-12f6-4ee6-867b-f0015ade00de-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:45:54.949452 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:54.949407 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmhks\" (UniqueName: \"kubernetes.io/projected/030973c9-12f6-4ee6-867b-f0015ade00de-kube-api-access-pmhks\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:45:55.406345 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.406306 2566 generic.go:356] "Generic (PLEG): container finished" podID="030973c9-12f6-4ee6-867b-f0015ade00de" containerID="257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb" exitCode=0 Jul 10 20:45:55.406787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.406353 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" Jul 10 20:45:55.406787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.406374 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" event={"ID":"030973c9-12f6-4ee6-867b-f0015ade00de","Type":"ContainerDied","Data":"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb"} Jul 10 20:45:55.406787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.406422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k" event={"ID":"030973c9-12f6-4ee6-867b-f0015ade00de","Type":"ContainerDied","Data":"bf0973cb96b3e3b2b5bae397a97af3311706d644f3f41a2d7d8d84bee61c4c1c"} Jul 10 20:45:55.406787 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.406444 2566 scope.go:117] "RemoveContainer" containerID="257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb" Jul 10 20:45:55.415893 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.415877 2566 scope.go:117] "RemoveContainer" containerID="257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb" Jul 10 20:45:55.416137 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:45:55.416119 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb\": container with ID starting with 257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb not found: ID does not exist" containerID="257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb" Jul 10 20:45:55.416214 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.416146 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb"} err="failed to get container status \"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb\": rpc error: code = NotFound desc = could not find container \"257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb\": container with ID starting with 257d3bf4d0536de8b414d7dfe65de2e5940c0ff4df0fd73409df890a48f8b8bb not found: ID does not exist" Jul 10 20:45:55.439008 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.438982 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:55.445680 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.445658 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zxd4k"] Jul 10 20:45:55.597264 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:45:55.597229 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030973c9-12f6-4ee6-867b-f0015ade00de" path="/var/lib/kubelet/pods/030973c9-12f6-4ee6-867b-f0015ade00de/volumes" Jul 10 20:46:09.852124 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:09.852093 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:09.950136 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:09.950103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:10.649451 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.649421 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:10.722226 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.722184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:10.921343 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.921255 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b"] Jul 10 20:46:10.921842 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.921824 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="030973c9-12f6-4ee6-867b-f0015ade00de" containerName="manager" Jul 10 20:46:10.921903 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.921844 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="030973c9-12f6-4ee6-867b-f0015ade00de" containerName="manager" Jul 10 20:46:10.921940 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.921914 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="030973c9-12f6-4ee6-867b-f0015ade00de" containerName="manager" Jul 10 20:46:10.948475 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.948445 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:10.949626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.949604 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b"] Jul 10 20:46:10.949827 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.949722 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:10.952771 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:10.952747 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x8bxq\"" Jul 10 20:46:11.100884 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.100841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.100884 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.100884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dfb\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-kube-api-access-54dfb\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.100962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101067 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101223 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101111 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101223 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.101223 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.101151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.201929 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201840 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.201929 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.201929 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201914 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201961 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.201979 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202054 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202210 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54dfb\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-kube-api-access-54dfb\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202515 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202362 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202515 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202685 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202663 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202729 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202666 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.202840 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.202790 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.204236 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.204214 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.204620 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.204601 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.211483 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.211459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.212000 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.211978 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dfb\" (UniqueName: \"kubernetes.io/projected/e28cb92f-9545-4df9-aef7-fbbc1d3642a8-kube-api-access-54dfb\") pod \"maas-default-gateway-openshift-default-56654dcf7-8629b\" (UID: \"e28cb92f-9545-4df9-aef7-fbbc1d3642a8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.261459 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.261428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:11.309823 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.309783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:11.412356 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.412331 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b"] Jul 10 20:46:11.413919 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:46:11.413893 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28cb92f_9545_4df9_aef7_fbbc1d3642a8.slice/crio-436562e9c98eb2302a6beddd01ea61311f3e972f3c4a526934059ec63188ef79 WatchSource:0}: Error finding container 436562e9c98eb2302a6beddd01ea61311f3e972f3c4a526934059ec63188ef79: Status 404 returned error can't find the container with id 436562e9c98eb2302a6beddd01ea61311f3e972f3c4a526934059ec63188ef79 Jul 10 20:46:11.416009 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.415972 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:46:11.416133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.416041 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:46:11.416133 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.416093 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:46:11.489650 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.489616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" event={"ID":"e28cb92f-9545-4df9-aef7-fbbc1d3642a8","Type":"ContainerStarted","Data":"bdc65db92508f7ac889b9ce1aac90051199de8e53c1102d2efdcbb576d775909"} Jul 10 20:46:11.489767 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.489683 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" event={"ID":"e28cb92f-9545-4df9-aef7-fbbc1d3642a8","Type":"ContainerStarted","Data":"436562e9c98eb2302a6beddd01ea61311f3e972f3c4a526934059ec63188ef79"} Jul 10 20:46:11.512647 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:11.512593 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" podStartSLOduration=1.512574844 podStartE2EDuration="1.512574844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:11.511121011 +0000 UTC m=+680.394952351" watchObservedRunningTime="2026-07-10 20:46:11.512574844 +0000 UTC m=+680.396406187" Jul 10 20:46:12.261714 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.261680 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:12.266608 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.266585 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:12.494606 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.494578 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:12.495408 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.495388 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:46:12.557629 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.556986 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:12.659061 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.659030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:12.756641 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:12.756611 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:13.433092 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.433063 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:13.451477 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.451451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:13.451646 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.451560 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:13.454183 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.454049 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pkph9\"" Jul 10 20:46:13.470255 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.470230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:13.525457 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.525425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2psgw\" (UniqueName: \"kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw\") pod \"authorino-58cd955675-tp5kf\" (UID: \"110e2ee7-e41d-4361-b585-c6cb3b6c9504\") " pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:13.593325 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.593278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:13.626991 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.626959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2psgw\" (UniqueName: \"kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw\") pod \"authorino-58cd955675-tp5kf\" (UID: \"110e2ee7-e41d-4361-b585-c6cb3b6c9504\") " pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:13.638760 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.638733 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2psgw\" (UniqueName: \"kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw\") pod \"authorino-58cd955675-tp5kf\" (UID: \"110e2ee7-e41d-4361-b585-c6cb3b6c9504\") " pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:13.765156 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.765129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:13.927832 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.927794 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:13.930064 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:46:13.930029 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod110e2ee7_e41d_4361_b585_c6cb3b6c9504.slice/crio-de4a977ab0f6730d09dbb65ff268f19658e7572db15fe0c75e07ad251db94f5a WatchSource:0}: Error finding container de4a977ab0f6730d09dbb65ff268f19658e7572db15fe0c75e07ad251db94f5a: Status 404 returned error can't find the container with id de4a977ab0f6730d09dbb65ff268f19658e7572db15fe0c75e07ad251db94f5a Jul 10 20:46:13.931485 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:13.931467 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 20:46:14.506262 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:14.506227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tp5kf" event={"ID":"110e2ee7-e41d-4361-b585-c6cb3b6c9504","Type":"ContainerStarted","Data":"de4a977ab0f6730d09dbb65ff268f19658e7572db15fe0c75e07ad251db94f5a"} Jul 10 20:46:17.525776 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:17.525730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tp5kf" event={"ID":"110e2ee7-e41d-4361-b585-c6cb3b6c9504","Type":"ContainerStarted","Data":"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2"} Jul 10 20:46:18.764853 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:18.764778 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tp5kf" podStartSLOduration=2.328464951 podStartE2EDuration="5.764754859s" totalImagesPullingTime="3.436289908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:46:17.549063306 +0000 UTC m=+686.432894646" watchObservedRunningTime="2026-07-10 20:46:18.764754859 +0000 UTC m=+687.648586204" Jul 10 20:46:18.767097 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:18.767069 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:19.536275 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:19.536218 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tp5kf" podUID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" containerName="authorino" containerID="cri-o://df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2" gracePeriod=30 Jul 10 20:46:19.791275 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:19.791223 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:19.892028 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:19.892002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2psgw\" (UniqueName: \"kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw\") pod \"110e2ee7-e41d-4361-b585-c6cb3b6c9504\" (UID: \"110e2ee7-e41d-4361-b585-c6cb3b6c9504\") " Jul 10 20:46:19.893852 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:19.893829 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw" (OuterVolumeSpecName: "kube-api-access-2psgw") pod "110e2ee7-e41d-4361-b585-c6cb3b6c9504" (UID: "110e2ee7-e41d-4361-b585-c6cb3b6c9504"). InnerVolumeSpecName "kube-api-access-2psgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:46:19.993255 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:19.993224 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2psgw\" (UniqueName: \"kubernetes.io/projected/110e2ee7-e41d-4361-b585-c6cb3b6c9504-kube-api-access-2psgw\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:46:20.542664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.542627 2566 generic.go:356] "Generic (PLEG): container finished" podID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" containerID="df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2" exitCode=0 Jul 10 20:46:20.542664 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.542670 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tp5kf" Jul 10 20:46:20.542885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.542708 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tp5kf" event={"ID":"110e2ee7-e41d-4361-b585-c6cb3b6c9504","Type":"ContainerDied","Data":"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2"} Jul 10 20:46:20.542885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.542751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tp5kf" event={"ID":"110e2ee7-e41d-4361-b585-c6cb3b6c9504","Type":"ContainerDied","Data":"de4a977ab0f6730d09dbb65ff268f19658e7572db15fe0c75e07ad251db94f5a"} Jul 10 20:46:20.542885 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.542768 2566 scope.go:117] "RemoveContainer" containerID="df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2" Jul 10 20:46:20.551688 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.551670 2566 scope.go:117] "RemoveContainer" containerID="df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2" Jul 10 20:46:20.551989 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:46:20.551972 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2\": container with ID starting with df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2 not found: ID does not exist" containerID="df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2" Jul 10 20:46:20.552066 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.552001 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2"} err="failed to get container status \"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2\": rpc error: code = NotFound desc = could not find container \"df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2\": container with ID starting with df64796b1b72db855cf89b34f717ef8f9320c9c23d4f3a8c049da5c01a0b93e2 not found: ID does not exist" Jul 10 20:46:20.572726 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.572695 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:20.577469 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:20.577442 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tp5kf"] Jul 10 20:46:21.596729 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:21.596697 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" path="/var/lib/kubelet/pods/110e2ee7-e41d-4361-b585-c6cb3b6c9504/volumes" Jul 10 20:46:22.008099 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:22.008066 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:22.100967 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:22.100928 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:22.200576 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:22.200542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:22.279240 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:22.279133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:33.766780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:33.766745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:33.867158 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:33.867123 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:34.223262 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:34.223165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:46.774032 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:46.773943 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:46.907879 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:46.907839 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:53.223974 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:53.223939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:53.332149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:53.332117 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:53.403780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:53.403744 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:53.520149 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:53.520115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:56.696396 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:56.696363 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:46:56.810821 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:46:56.810776 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:23.214601 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:23.214569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:30.404372 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:30.404339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:30.868406 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:30.868354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:31.038738 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:31.038695 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:31.371546 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:31.371497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:31.469221 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:31.469190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:39.946216 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.946182 2566 generic.go:356] "Generic (PLEG): container finished" podID="e28cb92f-9545-4df9-aef7-fbbc1d3642a8" containerID="bdc65db92508f7ac889b9ce1aac90051199de8e53c1102d2efdcbb576d775909" exitCode=0 Jul 10 20:47:39.946695 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.946252 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" event={"ID":"e28cb92f-9545-4df9-aef7-fbbc1d3642a8","Type":"ContainerDied","Data":"bdc65db92508f7ac889b9ce1aac90051199de8e53c1102d2efdcbb576d775909"} Jul 10 20:47:39.946695 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.946622 2566 scope.go:117] "RemoveContainer" containerID="bdc65db92508f7ac889b9ce1aac90051199de8e53c1102d2efdcbb576d775909" Jul 10 20:47:39.947609 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.947583 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:47:39.947716 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.947663 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:47:39.947775 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:39.947723 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 20:47:40.953315 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:40.953285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" event={"ID":"e28cb92f-9545-4df9-aef7-fbbc1d3642a8","Type":"ContainerStarted","Data":"d83df18551da8dd6842195df3a115f42e9b647ecb778f2bb01b3731211fa8509"} Jul 10 20:47:41.263755 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:41.263716 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:47:41.264199 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:41.264173 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8629b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 20:47:41.264400 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:41.264377 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" podUID="e28cb92f-9545-4df9-aef7-fbbc1d3642a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 20:47:41.681156 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:41.681051 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:41.803142 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:41.803106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:42.262464 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:42.262426 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8629b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 20:47:42.263502 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:42.263073 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" podUID="e28cb92f-9545-4df9-aef7-fbbc1d3642a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 20:47:42.494894 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:42.494861 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:47:43.262302 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:43.262264 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8629b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 20:47:43.262577 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:43.262547 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" podUID="e28cb92f-9545-4df9-aef7-fbbc1d3642a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 20:47:44.262031 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:44.261993 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-8629b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 10 20:47:44.262328 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:44.262297 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" podUID="e28cb92f-9545-4df9-aef7-fbbc1d3642a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 10 20:47:45.266315 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:45.266283 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:47:45.267407 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:45.267383 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8629b" Jul 10 20:47:52.227238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.227205 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb"] Jul 10 20:47:52.227759 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.227741 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" containerName="authorino" Jul 10 20:47:52.227759 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.227761 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" containerName="authorino" Jul 10 20:47:52.227935 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.227920 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="110e2ee7-e41d-4361-b585-c6cb3b6c9504" containerName="authorino" Jul 10 20:47:52.231013 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.230996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.234315 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.234293 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 20:47:52.234602 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.234584 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 20:47:52.235090 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.235071 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 20:47:52.235189 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.235079 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-szg4s\"" Jul 10 20:47:52.245818 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.245785 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb"] Jul 10 20:47:52.252462 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73b96c1-bbaa-4b32-9656-e504c0150ada-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.252569 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.252569 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252522 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.252569 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.252742 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.252742 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.252631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plvvj\" (UniqueName: \"kubernetes.io/projected/c73b96c1-bbaa-4b32-9656-e504c0150ada-kube-api-access-plvvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.281038 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.281008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:47:52.353418 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353418 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plvvj\" (UniqueName: \"kubernetes.io/projected/c73b96c1-bbaa-4b32-9656-e504c0150ada-kube-api-access-plvvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73b96c1-bbaa-4b32-9656-e504c0150ada-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353679 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353566 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353940 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.353990 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.353953 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.354037 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.354013 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.355774 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.355751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c73b96c1-bbaa-4b32-9656-e504c0150ada-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.356082 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.356063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c73b96c1-bbaa-4b32-9656-e504c0150ada-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.366238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.366216 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plvvj\" (UniqueName: \"kubernetes.io/projected/c73b96c1-bbaa-4b32-9656-e504c0150ada-kube-api-access-plvvj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb\" (UID: \"c73b96c1-bbaa-4b32-9656-e504c0150ada\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.541256 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.541215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:47:52.686663 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:52.686494 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb"] Jul 10 20:47:52.687154 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:47:52.687119 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73b96c1_bbaa_4b32_9656_e504c0150ada.slice/crio-1d0b0dcb16f0fe5aba03ddc5ee90b971923dfc1bb858fb1a78ae50655169a305 WatchSource:0}: Error finding container 1d0b0dcb16f0fe5aba03ddc5ee90b971923dfc1bb858fb1a78ae50655169a305: Status 404 returned error can't find the container with id 1d0b0dcb16f0fe5aba03ddc5ee90b971923dfc1bb858fb1a78ae50655169a305 Jul 10 20:47:53.022622 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:53.022569 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" event={"ID":"c73b96c1-bbaa-4b32-9656-e504c0150ada","Type":"ContainerStarted","Data":"1d0b0dcb16f0fe5aba03ddc5ee90b971923dfc1bb858fb1a78ae50655169a305"} Jul 10 20:47:54.029917 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:54.029879 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" event={"ID":"c73b96c1-bbaa-4b32-9656-e504c0150ada","Type":"ContainerStarted","Data":"47767e840ed2e8bee628d9b211e33dedddfdb797a90fe1b2fd335f733cd19020"} Jul 10 20:47:54.060070 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:47:54.060018 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" podStartSLOduration=0.928141715 podStartE2EDuration="2.060000729s" totalImagesPullingTime="1.131859014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:47:54.059523655 +0000 UTC m=+782.943354998" watchObservedRunningTime="2026-07-10 20:47:54.060000729 +0000 UTC m=+782.943832072" Jul 10 20:48:02.542239 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:02.542206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:48:02.542795 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:02.542312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:48:02.555120 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:02.555088 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:48:02.566083 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:02.566054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:03.090985 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:03.090959 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb" Jul 10 20:48:08.462881 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:08.462850 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:13.443067 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:13.442989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:29.074085 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.074053 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48"] Jul 10 20:48:29.079430 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.079400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.083446 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.083402 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 20:48:29.105516 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.105483 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48"] Jul 10 20:48:29.130137 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.130102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:48:29.216350 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216309 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.216570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.216570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216433 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.216570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.216570 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zld4j\" (UniqueName: \"kubernetes.io/projected/39498b6f-2ba7-401d-849f-1b123571b4c5-kube-api-access-zld4j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.216869 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.216844 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39498b6f-2ba7-401d-849f-1b123571b4c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317525 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39498b6f-2ba7-401d-849f-1b123571b4c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317543 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317567 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.317704 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.317655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zld4j\" (UniqueName: \"kubernetes.io/projected/39498b6f-2ba7-401d-849f-1b123571b4c5-kube-api-access-zld4j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.318122 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.318093 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.318238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.318125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.318238 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.318170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.319867 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.319849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39498b6f-2ba7-401d-849f-1b123571b4c5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.320137 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.320121 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39498b6f-2ba7-401d-849f-1b123571b4c5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.327533 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.327466 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zld4j\" (UniqueName: \"kubernetes.io/projected/39498b6f-2ba7-401d-849f-1b123571b4c5-kube-api-access-zld4j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lvn48\" (UID: \"39498b6f-2ba7-401d-849f-1b123571b4c5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.391523 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.391491 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:29.530639 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:29.530613 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48"] Jul 10 20:48:29.532661 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:48:29.532631 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39498b6f_2ba7_401d_849f_1b123571b4c5.slice/crio-f7628cecbff83fb0d29d6e0b82f0e98652e995631207f05ae577e0329697f128 WatchSource:0}: Error finding container f7628cecbff83fb0d29d6e0b82f0e98652e995631207f05ae577e0329697f128: Status 404 returned error can't find the container with id f7628cecbff83fb0d29d6e0b82f0e98652e995631207f05ae577e0329697f128 Jul 10 20:48:30.230172 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:30.230125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" event={"ID":"39498b6f-2ba7-401d-849f-1b123571b4c5","Type":"ContainerStarted","Data":"ff18d73c466f308ac26578200c8b12972f6eed915108ce43ef292b54a795e702"} Jul 10 20:48:30.230626 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:30.230193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" event={"ID":"39498b6f-2ba7-401d-849f-1b123571b4c5","Type":"ContainerStarted","Data":"f7628cecbff83fb0d29d6e0b82f0e98652e995631207f05ae577e0329697f128"} Jul 10 20:48:30.260271 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:30.260213 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" podStartSLOduration=1.084199169 podStartE2EDuration="1.26019629s" totalImagesPullingTime="175.997121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:48:30.256947294 +0000 UTC m=+819.140778638" watchObservedRunningTime="2026-07-10 20:48:30.26019629 +0000 UTC m=+819.144027633" Jul 10 20:48:39.391894 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:39.391854 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:39.392496 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:39.391925 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:39.404376 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:39.404349 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:40.292694 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:40.292666 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lvn48" Jul 10 20:48:47.661068 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:48:47.661020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:48.219290 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:48.219208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:49.437511 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:49.437476 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:49:51.549128 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:51.549101 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:49:51.550058 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:51.550038 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:49:51.553402 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:51.553385 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:49:51.554121 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:49:51.554102 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:50:02.365917 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.365878 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d999c77f5-bbbtt"] Jul 10 20:50:02.369649 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.369630 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.372802 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.372758 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pkph9\"" Jul 10 20:50:02.372968 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.372758 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 20:50:02.380165 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.380138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d999c77f5-bbbtt"] Jul 10 20:50:02.426778 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.426735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvzb8\" (UniqueName: \"kubernetes.io/projected/6723c0cf-2aac-47da-ab18-d91606325dd2-kube-api-access-rvzb8\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.427012 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.426801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6723c0cf-2aac-47da-ab18-d91606325dd2-tls-cert\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.527774 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.527735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6723c0cf-2aac-47da-ab18-d91606325dd2-tls-cert\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.528002 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.527883 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvzb8\" (UniqueName: \"kubernetes.io/projected/6723c0cf-2aac-47da-ab18-d91606325dd2-kube-api-access-rvzb8\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.530203 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.530181 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6723c0cf-2aac-47da-ab18-d91606325dd2-tls-cert\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.550747 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.550710 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvzb8\" (UniqueName: \"kubernetes.io/projected/6723c0cf-2aac-47da-ab18-d91606325dd2-kube-api-access-rvzb8\") pod \"authorino-d999c77f5-bbbtt\" (UID: \"6723c0cf-2aac-47da-ab18-d91606325dd2\") " pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.679761 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.679687 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d999c77f5-bbbtt" Jul 10 20:50:02.730584 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.730546 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:02.822924 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.822891 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d999c77f5-bbbtt"] Jul 10 20:50:02.824062 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:02.824013 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:02.825651 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:50:02.825616 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6723c0cf_2aac_47da_ab18_d91606325dd2.slice/crio-64242dcedc2e340bceaa8f2a410cf7f2590b0a632c4cfef253d9ad63bc41a216 WatchSource:0}: Error finding container 64242dcedc2e340bceaa8f2a410cf7f2590b0a632c4cfef253d9ad63bc41a216: Status 404 returned error can't find the container with id 64242dcedc2e340bceaa8f2a410cf7f2590b0a632c4cfef253d9ad63bc41a216 Jul 10 20:50:03.720155 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:03.720110 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d999c77f5-bbbtt" event={"ID":"6723c0cf-2aac-47da-ab18-d91606325dd2","Type":"ContainerStarted","Data":"8930feb15f9d7d5a107a3edb5f28b253886b679c9afba6ad77604844054bdf8f"} Jul 10 20:50:03.720155 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:03.720169 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d999c77f5-bbbtt" event={"ID":"6723c0cf-2aac-47da-ab18-d91606325dd2","Type":"ContainerStarted","Data":"64242dcedc2e340bceaa8f2a410cf7f2590b0a632c4cfef253d9ad63bc41a216"} Jul 10 20:50:03.738082 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:03.738030 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d999c77f5-bbbtt" podStartSLOduration=1.014683983 podStartE2EDuration="1.738008835s" totalImagesPullingTime="723.324852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:50:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:50:03.735712625 +0000 UTC m=+912.619543968" watchObservedRunningTime="2026-07-10 20:50:03.738008835 +0000 UTC m=+912.621840248" Jul 10 20:50:03.938412 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:03.938328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:04.035945 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:04.035911 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:50:21.345391 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:50:21.345357 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:14.160460 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:14.160381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:15.521688 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:15.521650 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:16.549026 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:16.548996 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:51.699168 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:51.699128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:51.760680 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:51.760649 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:55.040204 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:55.040168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:55.143366 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:55.143310 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:51:55.246159 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:51:55.246125 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:01.004784 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:01.004732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:01.066166 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:01.066128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:01.150423 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:01.150391 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:01.247932 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:01.247900 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:01.385494 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:01.385381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:07.064426 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:07.064386 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:07.163667 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:07.163600 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:17.120373 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:17.120340 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:17.220583 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:17.220554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:52:39.507909 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:52:39.507870 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:54:51.585077 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:54:51.585041 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:54:51.588659 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:54:51.588636 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:54:51.590137 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:54:51.590115 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:54:51.593663 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:54:51.593641 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:55:23.486305 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:55:23.486261 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:55:25.103801 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:55:25.103763 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:56:05.656669 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:05.656527 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:56:05.657319 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:05.656881 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" podUID="d668e025-57b7-4109-8fe3-0823a0156319" containerName="manager" containerID="cri-o://bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806" gracePeriod=10 Jul 10 20:56:09.514979 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.514917 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:56:09.684651 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.684613 2566 generic.go:356] "Generic (PLEG): container finished" podID="d668e025-57b7-4109-8fe3-0823a0156319" containerID="bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806" exitCode=0 Jul 10 20:56:09.684651 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.684654 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" Jul 10 20:56:09.684883 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.684654 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" event={"ID":"d668e025-57b7-4109-8fe3-0823a0156319","Type":"ContainerDied","Data":"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806"} Jul 10 20:56:09.684883 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.684788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7" event={"ID":"d668e025-57b7-4109-8fe3-0823a0156319","Type":"ContainerDied","Data":"73650910eae1def1cde80cfe4e435bfe258835808bf6ce34de078e19f5b10f25"} Jul 10 20:56:09.684883 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.684837 2566 scope.go:117] "RemoveContainer" containerID="bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806" Jul 10 20:56:09.687625 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.687607 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume\") pod \"d668e025-57b7-4109-8fe3-0823a0156319\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " Jul 10 20:56:09.687735 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.687660 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f78pt\" (UniqueName: \"kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt\") pod \"d668e025-57b7-4109-8fe3-0823a0156319\" (UID: \"d668e025-57b7-4109-8fe3-0823a0156319\") " Jul 10 20:56:09.688030 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.688006 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d668e025-57b7-4109-8fe3-0823a0156319" (UID: "d668e025-57b7-4109-8fe3-0823a0156319"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 20:56:09.689941 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.689910 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt" (OuterVolumeSpecName: "kube-api-access-f78pt") pod "d668e025-57b7-4109-8fe3-0823a0156319" (UID: "d668e025-57b7-4109-8fe3-0823a0156319"). InnerVolumeSpecName "kube-api-access-f78pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 20:56:09.694467 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.694448 2566 scope.go:117] "RemoveContainer" containerID="bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806" Jul 10 20:56:09.694731 ip-10-0-137-185 kubenswrapper[2566]: E0710 20:56:09.694713 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806\": container with ID starting with bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806 not found: ID does not exist" containerID="bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806" Jul 10 20:56:09.694779 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.694740 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806"} err="failed to get container status \"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806\": rpc error: code = NotFound desc = could not find container \"bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806\": container with ID starting with bfe89cc3dd43cf043152c9311137a6d1252c2294594d5d88f5cc7c9fefa71806 not found: ID does not exist" Jul 10 20:56:09.789736 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.789641 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d668e025-57b7-4109-8fe3-0823a0156319-extensions-socket-volume\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:56:09.789736 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:09.789670 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f78pt\" (UniqueName: \"kubernetes.io/projected/d668e025-57b7-4109-8fe3-0823a0156319-kube-api-access-f78pt\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 20:56:10.012096 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:10.012060 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:56:10.016122 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:10.016095 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tlkl7"] Jul 10 20:56:11.596905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:11.596872 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d668e025-57b7-4109-8fe3-0823a0156319" path="/var/lib/kubelet/pods/d668e025-57b7-4109-8fe3-0823a0156319/volumes" Jul 10 20:56:12.239158 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:56:12.239125 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 20:57:06.800354 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.800308 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8"] Jul 10 20:57:06.800864 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.800846 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d668e025-57b7-4109-8fe3-0823a0156319" containerName="manager" Jul 10 20:57:06.800864 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.800861 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d668e025-57b7-4109-8fe3-0823a0156319" containerName="manager" Jul 10 20:57:06.800951 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.800936 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d668e025-57b7-4109-8fe3-0823a0156319" containerName="manager" Jul 10 20:57:06.805390 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.805375 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:06.807655 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.807628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-94hz6\"" Jul 10 20:57:06.817780 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.817756 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8"] Jul 10 20:57:06.955905 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.955866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqvkf\" (UniqueName: \"kubernetes.io/projected/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-kube-api-access-zqvkf\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:06.956084 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:06.955920 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.057492 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.057392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqvkf\" (UniqueName: \"kubernetes.io/projected/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-kube-api-access-zqvkf\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.057492 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.057453 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.057871 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.057852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.065745 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.065721 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqvkf\" (UniqueName: \"kubernetes.io/projected/f348749e-53a1-4882-bd4d-6bc6f8ae9bf4-kube-api-access-zqvkf\") pod \"kuadrant-operator-controller-manager-86878c769f-sl2x8\" (UID: \"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.115835 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.115774 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:07.282870 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:07.282843 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8"] Jul 10 20:57:07.283440 ip-10-0-137-185 kubenswrapper[2566]: W0710 20:57:07.283409 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf348749e_53a1_4882_bd4d_6bc6f8ae9bf4.slice/crio-a03d1a2e8f18058cd8cb94095cb9898a4effb22af04e9a9c29e3629b653d3374 WatchSource:0}: Error finding container a03d1a2e8f18058cd8cb94095cb9898a4effb22af04e9a9c29e3629b653d3374: Status 404 returned error can't find the container with id a03d1a2e8f18058cd8cb94095cb9898a4effb22af04e9a9c29e3629b653d3374 Jul 10 20:57:08.011656 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:08.011614 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" event={"ID":"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4","Type":"ContainerStarted","Data":"9322499ea9053580e66d22948bf82e10a375548636e835b2b03d8456439d518a"} Jul 10 20:57:08.012088 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:08.011674 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" event={"ID":"f348749e-53a1-4882-bd4d-6bc6f8ae9bf4","Type":"ContainerStarted","Data":"a03d1a2e8f18058cd8cb94095cb9898a4effb22af04e9a9c29e3629b653d3374"} Jul 10 20:57:08.012088 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:08.011696 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:57:08.032511 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:08.032461 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" podStartSLOduration=2.032444842 podStartE2EDuration="2.032444842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:57:08.030653237 +0000 UTC m=+1336.914484574" watchObservedRunningTime="2026-07-10 20:57:08.032444842 +0000 UTC m=+1336.916276185" Jul 10 20:57:19.019209 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:57:19.019124 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sl2x8" Jul 10 20:59:51.620844 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:59:51.620819 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:59:51.625317 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:59:51.625293 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 20:59:51.627240 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:59:51.627220 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 20:59:51.631266 ip-10-0-137-185 kubenswrapper[2566]: I0710 20:59:51.631249 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:00:12.995846 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:00:12.995794 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:01:24.695138 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:01:24.695097 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:02:57.261514 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:02:57.261485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:04.839653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:04.839609 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:13.080194 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:13.080106 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:18.136118 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:18.136085 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:21.014442 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:21.014405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:21.113425 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:21.113386 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:21.498496 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:21.498459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:27.325238 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.325200 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:27.429238 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.429203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:27.512107 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.512074 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:27.516177 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.516156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.518475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.518459 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6882a6b6-openshift-default-dockercfg-j7xmv\"" Jul 10 21:03:27.538045 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.538024 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:27.552061 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.552035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:27.622766 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.622681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdjps\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.622766 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.622743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623067 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.622774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623067 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.622877 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623067 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.622906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623067 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.623030 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623223 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.623083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623223 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.623119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.623311 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.623241 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.705086 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.705052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:27.724301 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724458 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdjps\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724522 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724587 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724670 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724735 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724697 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724788 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724890 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.724890 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.725008 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.725008 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.724907 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.725162 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.725138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.725274 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.725204 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.725545 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.725527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.727022 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.726999 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.727201 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.727183 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.739513 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.739490 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.742547 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.742523 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdjps\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps\") pod \"e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.829913 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.829882 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:27.972528 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.972497 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:27.974249 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:03:27.974223 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a601e2_2be6_48fb_a504_7c8acdd4f1a1.slice/crio-0a51029e09158fb7c1a3f18c8ab3d27495f7091e7caf6cb49f9bfb8cf26a0eb7 WatchSource:0}: Error finding container 0a51029e09158fb7c1a3f18c8ab3d27495f7091e7caf6cb49f9bfb8cf26a0eb7: Status 404 returned error can't find the container with id 0a51029e09158fb7c1a3f18c8ab3d27495f7091e7caf6cb49f9bfb8cf26a0eb7 Jul 10 21:03:27.976451 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.976422 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:27.976534 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.976488 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:27.976534 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:27.976516 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:28.089883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.089843 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" event={"ID":"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1","Type":"ContainerStarted","Data":"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58"} Jul 10 21:03:28.090050 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.089913 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" event={"ID":"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1","Type":"ContainerStarted","Data":"0a51029e09158fb7c1a3f18c8ab3d27495f7091e7caf6cb49f9bfb8cf26a0eb7"} Jul 10 21:03:28.110383 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.110329 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" podStartSLOduration=1.110312394 podStartE2EDuration="1.110312394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:28.10805942 +0000 UTC m=+1716.991890786" watchObservedRunningTime="2026-07-10 21:03:28.110312394 +0000 UTC m=+1716.994143738" Jul 10 21:03:28.732762 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.732719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:28.824377 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.824343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:28.830193 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.830171 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:28.893198 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.893162 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:28.994126 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:28.994096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:29.402096 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:29.402009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:29.502341 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:29.502308 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:29.835594 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:29.835564 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:30.101358 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:30.101283 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:30.102330 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:30.102314 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:31.316595 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:31.316561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:33.720200 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:33.720167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:33.874747 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:33.874706 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:33.874977 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:33.874949 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" podUID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" containerName="istio-proxy" containerID="cri-o://7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58" gracePeriod=30 Jul 10 21:03:33.904315 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:33.904285 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:33.999199 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:33.999152 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:34.127379 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:34.127348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:36.400284 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:36.400252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:39.143987 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.143925 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:39.149560 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.149535 2566 generic.go:356] "Generic (PLEG): container finished" podID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" containerID="7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58" exitCode=0 Jul 10 21:03:39.149667 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.149585 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" Jul 10 21:03:39.149667 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.149619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" event={"ID":"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1","Type":"ContainerDied","Data":"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58"} Jul 10 21:03:39.149743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.149675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx" event={"ID":"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1","Type":"ContainerDied","Data":"0a51029e09158fb7c1a3f18c8ab3d27495f7091e7caf6cb49f9bfb8cf26a0eb7"} Jul 10 21:03:39.149743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.149705 2566 scope.go:117] "RemoveContainer" containerID="7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58" Jul 10 21:03:39.158115 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.158099 2566 scope.go:117] "RemoveContainer" containerID="7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58" Jul 10 21:03:39.158376 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:03:39.158358 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58\": container with ID starting with 7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58 not found: ID does not exist" containerID="7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58" Jul 10 21:03:39.158450 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.158387 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58"} err="failed to get container status \"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58\": rpc error: code = NotFound desc = could not find container \"7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58\": container with ID starting with 7a12a15477034b86386dc4f4ca63516eff584bce62f8f0101c2cc469bfab3c58 not found: ID does not exist" Jul 10 21:03:39.234083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234055 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234250 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234095 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234250 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234114 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234250 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234135 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdjps\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234250 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234163 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234250 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234205 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234532 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234254 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234532 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234385 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:39.234532 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234407 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234532 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234458 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs\") pod \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\" (UID: \"a5a601e2-2be6-48fb-a504-7c8acdd4f1a1\") " Jul 10 21:03:39.234740 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234566 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:39.234740 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234678 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:03:39.234880 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234800 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:39.234974 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234956 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.235029 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234981 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.235029 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.234996 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.235029 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.235009 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.235187 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.235088 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:39.236501 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.236465 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:03:39.236628 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.236499 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:39.236726 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.236701 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps" (OuterVolumeSpecName: "kube-api-access-sdjps") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "kube-api-access-sdjps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:39.236879 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.236860 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" (UID: "a5a601e2-2be6-48fb-a504-7c8acdd4f1a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:39.335681 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.335652 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.335681 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.335677 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.335681 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.335688 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdjps\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-kube-api-access-sdjps\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.335918 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.335698 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.335918 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.335706 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:39.482282 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.482252 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:39.492395 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.492362 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6882a6b6-openshift-default-56649744d9-8b8lx"] Jul 10 21:03:39.598632 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.598599 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" path="/var/lib/kubelet/pods/a5a601e2-2be6-48fb-a504-7c8acdd4f1a1/volumes" Jul 10 21:03:39.849060 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.849025 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:39.950411 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.950375 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:39.994575 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.994547 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:39.995106 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.995084 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" containerName="istio-proxy" Jul 10 21:03:39.995106 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.995104 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" containerName="istio-proxy" Jul 10 21:03:39.995324 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:39.995227 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a601e2-2be6-48fb-a504-7c8acdd4f1a1" containerName="istio-proxy" Jul 10 21:03:40.000073 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.000050 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.002839 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.002799 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2c8fd457-openshift-default-dockercfg-2thvn\"" Jul 10 21:03:40.011374 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.011346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:40.038985 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.038960 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:40.043225 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043346 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043346 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043333 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043462 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043462 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45kst\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043566 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043621 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043621 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.043727 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.043656 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.116344 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.116267 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:40.144763 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.144967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145002 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45kst\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145554 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145554 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145346 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145554 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145382 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145554 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145423 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.145839 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.145823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.147223 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.147196 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.147424 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.147404 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.153049 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.153016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45kst\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.153293 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.153276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token\") pod \"e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.313225 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.313185 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:40.459308 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.459277 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:40.462757 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:03:40.462717 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32af0e78_25ed_4747_955c_f891d2558d84.slice/crio-76a3ca9bf2b6cd375ebb612b2e4f8f23ad3e33aeb55e80e3539467b9312b0330 WatchSource:0}: Error finding container 76a3ca9bf2b6cd375ebb612b2e4f8f23ad3e33aeb55e80e3539467b9312b0330: Status 404 returned error can't find the container with id 76a3ca9bf2b6cd375ebb612b2e4f8f23ad3e33aeb55e80e3539467b9312b0330 Jul 10 21:03:40.465613 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.465574 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:40.465721 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.465659 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:40.465721 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:40.465701 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:41.166104 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.166061 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerStarted","Data":"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc"} Jul 10 21:03:41.166604 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.166136 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerStarted","Data":"76a3ca9bf2b6cd375ebb612b2e4f8f23ad3e33aeb55e80e3539467b9312b0330"} Jul 10 21:03:41.197515 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.197463 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" podStartSLOduration=2.197446492 podStartE2EDuration="2.197446492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:41.195083841 +0000 UTC m=+1730.078915184" watchObservedRunningTime="2026-07-10 21:03:41.197446492 +0000 UTC m=+1730.081277835" Jul 10 21:03:41.233173 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.233139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:41.313901 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.313866 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:41.319044 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.319019 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:41.327115 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.327058 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:41.496018 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.495974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:41.898140 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.898042 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:41.998399 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:41.998367 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:42.099081 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:42.099045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:42.172519 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:42.172435 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:42.173580 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:42.173558 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:43.259003 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:43.258967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:46.262322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:46.262260 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:46.633091 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:46.632990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:52.231264 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.231162 2566 generic.go:356] "Generic (PLEG): container finished" podID="32af0e78-25ed-4747-955c-f891d2558d84" containerID="b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc" exitCode=0 Jul 10 21:03:52.231264 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.231240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerDied","Data":"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc"} Jul 10 21:03:52.231753 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.231738 2566 scope.go:117] "RemoveContainer" containerID="b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc" Jul 10 21:03:52.232909 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.232880 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:52.233008 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.232961 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:52.233052 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.233007 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:52.324060 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.324026 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:52.345069 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.345041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:52.448444 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:52.448410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:53.238523 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:53.238488 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerStarted","Data":"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a"} Jul 10 21:03:53.239144 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:53.238549 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" containerID="cri-o://3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a" gracePeriod=30 Jul 10 21:03:54.896790 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:54.896758 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:57.172495 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:57.172457 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:58.069133 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.069100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:58.172488 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.172450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:58.240167 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.240137 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:03:58.243866 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.243839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.246422 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.246399 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e9b5f3ba-openshift-default-dockercfg-cnlh7\"" Jul 10 21:03:58.256695 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.256670 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:03:58.284047 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.284016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:58.331179 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331323 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4bkg\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331323 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331323 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331480 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331480 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331480 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331433 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331594 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.331594 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.331515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.382696 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.382627 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:58.432653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.432791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432663 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.432791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.432791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.432791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432747 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433071 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433071 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.432942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433071 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bkg\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433233 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433233 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433184 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433471 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433452 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433636 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.433774 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.434002 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.433982 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.435233 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.435212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.435350 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.435324 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.440989 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.440961 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.441066 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.440967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4bkg\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg\") pod \"e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.512825 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.512788 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:58.621318 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.621234 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:58.634440 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634413 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45kst\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634499 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634615 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634599 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634674 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634642 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634747 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634725 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634820 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634758 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634820 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634795 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634934 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634826 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:58.634934 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634856 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:58.634934 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634871 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.634934 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.634900 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token\") pod \"32af0e78-25ed-4747-955c-f891d2558d84\" (UID: \"32af0e78-25ed-4747-955c-f891d2558d84\") " Jul 10 21:03:58.635134 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635100 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:03:58.635300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635277 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.635443 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635306 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.635443 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635322 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32af0e78-25ed-4747-955c-f891d2558d84-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.635552 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635516 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:58.635728 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.635701 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data" (OuterVolumeSpecName: "istio-data") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:58.637063 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.637040 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst" (OuterVolumeSpecName: "kube-api-access-45kst") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "kube-api-access-45kst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:58.637340 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.637306 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:03:58.637539 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.637521 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:03:58.637619 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.637601 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token" (OuterVolumeSpecName: "istio-token") pod "32af0e78-25ed-4747-955c-f891d2558d84" (UID: "32af0e78-25ed-4747-955c-f891d2558d84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736833 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32af0e78-25ed-4747-955c-f891d2558d84-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736866 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736884 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736897 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32af0e78-25ed-4747-955c-f891d2558d84-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736910 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.736982 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.736924 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45kst\" (UniqueName: \"kubernetes.io/projected/32af0e78-25ed-4747-955c-f891d2558d84-kube-api-access-45kst\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:03:58.759185 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.759156 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:03:58.761075 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:03:58.761052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f20a28_8f9b_466b_9cf2_f50ca66da601.slice/crio-44978959051f86759ee94da83b5222514fd4bd04ec35bf6e791bcd1a0b736e30 WatchSource:0}: Error finding container 44978959051f86759ee94da83b5222514fd4bd04ec35bf6e791bcd1a0b736e30: Status 404 returned error can't find the container with id 44978959051f86759ee94da83b5222514fd4bd04ec35bf6e791bcd1a0b736e30 Jul 10 21:03:58.763451 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.763419 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:58.763514 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.763499 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:58.763558 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:58.763544 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:03:59.272761 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.272728 2566 generic.go:356] "Generic (PLEG): container finished" podID="32af0e78-25ed-4747-955c-f891d2558d84" containerID="3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a" exitCode=0 Jul 10 21:03:59.273200 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.272784 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" Jul 10 21:03:59.273200 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.272822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerDied","Data":"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a"} Jul 10 21:03:59.273200 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.272889 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc" event={"ID":"32af0e78-25ed-4747-955c-f891d2558d84","Type":"ContainerDied","Data":"76a3ca9bf2b6cd375ebb612b2e4f8f23ad3e33aeb55e80e3539467b9312b0330"} Jul 10 21:03:59.273200 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.272923 2566 scope.go:117] "RemoveContainer" containerID="3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a" Jul 10 21:03:59.274563 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.274522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" event={"ID":"40f20a28-8f9b-466b-9cf2-f50ca66da601","Type":"ContainerStarted","Data":"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0"} Jul 10 21:03:59.274689 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.274593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" event={"ID":"40f20a28-8f9b-466b-9cf2-f50ca66da601","Type":"ContainerStarted","Data":"44978959051f86759ee94da83b5222514fd4bd04ec35bf6e791bcd1a0b736e30"} Jul 10 21:03:59.288256 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.288234 2566 scope.go:117] "RemoveContainer" containerID="b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc" Jul 10 21:03:59.296829 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.296797 2566 scope.go:117] "RemoveContainer" containerID="3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a" Jul 10 21:03:59.297059 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:03:59.297045 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a\": container with ID starting with 3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a not found: ID does not exist" containerID="3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a" Jul 10 21:03:59.297105 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.297068 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a"} err="failed to get container status \"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a\": rpc error: code = NotFound desc = could not find container \"3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a\": container with ID starting with 3971574565d901d3335a532264db1c9200ec1337aaf3dfdcf15fdc1c96c5dd1a not found: ID does not exist" Jul 10 21:03:59.297105 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.297085 2566 scope.go:117] "RemoveContainer" containerID="b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc" Jul 10 21:03:59.297305 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:03:59.297286 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc\": container with ID starting with b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc not found: ID does not exist" containerID="b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc" Jul 10 21:03:59.297344 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.297313 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc"} err="failed to get container status \"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc\": rpc error: code = NotFound desc = could not find container \"b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc\": container with ID starting with b528c9ef771e8abaf2f754011993d9f794539c9f25e1da3d1b5bfd0bd030d0dc not found: ID does not exist" Jul 10 21:03:59.303436 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.303394 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" podStartSLOduration=1.303375882 podStartE2EDuration="1.303375882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:03:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:03:59.300741566 +0000 UTC m=+1748.184572936" watchObservedRunningTime="2026-07-10 21:03:59.303375882 +0000 UTC m=+1748.187207225" Jul 10 21:03:59.317068 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.317045 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:59.320781 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.320758 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2c8fd457-openshift-default-9c5f8f78f-fm2sc"] Jul 10 21:03:59.597498 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.597420 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32af0e78-25ed-4747-955c-f891d2558d84" path="/var/lib/kubelet/pods/32af0e78-25ed-4747-955c-f891d2558d84/volumes" Jul 10 21:03:59.622026 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.621995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:59.626776 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.626752 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:03:59.851884 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.851781 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:03:59.958655 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:03:59.958625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:00.088002 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:00.087967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:00.189124 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:00.189040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:00.281915 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:00.281889 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:04:00.282785 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:00.282769 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:04:01.647737 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:01.647706 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:03.840177 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:03.840141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:03.964563 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:03.964518 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:04:03.964748 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:03.964714 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" podUID="40f20a28-8f9b-466b-9cf2-f50ca66da601" containerName="istio-proxy" containerID="cri-o://357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0" gracePeriod=30 Jul 10 21:04:03.989757 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:03.989729 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:04.100497 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:04.100414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:04.198171 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:04.198137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:05.649589 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:05.649558 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:09.219575 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.219544 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:04:09.333399 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.333362 2566 generic.go:356] "Generic (PLEG): container finished" podID="40f20a28-8f9b-466b-9cf2-f50ca66da601" containerID="357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0" exitCode=0 Jul 10 21:04:09.333578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.333412 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" Jul 10 21:04:09.333578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.333451 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" event={"ID":"40f20a28-8f9b-466b-9cf2-f50ca66da601","Type":"ContainerDied","Data":"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0"} Jul 10 21:04:09.333578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.333542 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw" event={"ID":"40f20a28-8f9b-466b-9cf2-f50ca66da601","Type":"ContainerDied","Data":"44978959051f86759ee94da83b5222514fd4bd04ec35bf6e791bcd1a0b736e30"} Jul 10 21:04:09.333578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.333565 2566 scope.go:117] "RemoveContainer" containerID="357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0" Jul 10 21:04:09.342876 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.342859 2566 scope.go:117] "RemoveContainer" containerID="357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0" Jul 10 21:04:09.343135 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:04:09.343116 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0\": container with ID starting with 357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0 not found: ID does not exist" containerID="357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0" Jul 10 21:04:09.343219 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.343145 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0"} err="failed to get container status \"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0\": rpc error: code = NotFound desc = could not find container \"357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0\": container with ID starting with 357b32d5e33e1feb031bb61f8a14c9138c9051508f32411bdf6b808330257cc0 not found: ID does not exist" Jul 10 21:04:09.344375 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344356 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4bkg\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344459 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344445 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344498 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344482 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344540 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344513 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344576 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344551 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344626 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344759 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344737 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344861 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344793 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.344916 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344736 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data" (OuterVolumeSpecName: "istio-data") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:09.344916 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344850 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:09.344916 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344862 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket\") pod \"40f20a28-8f9b-466b-9cf2-f50ca66da601\" (UID: \"40f20a28-8f9b-466b-9cf2-f50ca66da601\") " Jul 10 21:04:09.345080 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.344957 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:04:09.345080 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345027 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:09.345241 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345219 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:09.345507 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345485 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.345551 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345515 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.345551 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345532 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.345551 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345548 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40f20a28-8f9b-466b-9cf2-f50ca66da601-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.345647 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.345564 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.346619 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.346590 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg" (OuterVolumeSpecName: "kube-api-access-v4bkg") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "kube-api-access-v4bkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:09.346747 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.346724 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:04:09.346945 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.346925 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:04:09.347094 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.347071 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token" (OuterVolumeSpecName: "istio-token") pod "40f20a28-8f9b-466b-9cf2-f50ca66da601" (UID: "40f20a28-8f9b-466b-9cf2-f50ca66da601"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:04:09.446546 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.446509 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.446546 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.446539 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4bkg\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-kube-api-access-v4bkg\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.446546 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.446550 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.446546 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.446558 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40f20a28-8f9b-466b-9cf2-f50ca66da601-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:04:09.650281 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.650247 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:04:09.653950 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.653925 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e9b5f3ba-openshift-default-688879cf67-l7qpw"] Jul 10 21:04:09.845421 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.845390 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:09.914051 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:09.913964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:10.394167 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:10.394131 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:10.494838 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:10.494772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:11.597271 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:11.597228 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40f20a28-8f9b-466b-9cf2-f50ca66da601" path="/var/lib/kubelet/pods/40f20a28-8f9b-466b-9cf2-f50ca66da601/volumes" Jul 10 21:04:11.895792 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:11.895708 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:18.574874 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:18.574839 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:18.949688 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:18.949604 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:19.051769 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:19.051733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:29.795621 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:29.795583 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:45.024933 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:45.024843 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:50.746532 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:50.746495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:50.822024 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:50.821991 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:50.925593 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:50.925555 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:04:51.654866 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:51.654839 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:04:51.659091 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:51.659060 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:04:51.662860 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:51.662838 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:04:51.667957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:04:51.667939 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:05:06.817194 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:06.817155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:12.651154 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:12.651116 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:12.756576 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:12.756535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:12.883425 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:12.883390 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:23.356636 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:23.356601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:24.152343 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:24.152305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:29.991271 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:29.991230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:30.110039 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:30.109993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:30.212007 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:30.211972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:30.313183 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:30.313102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:31.109075 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:31.109022 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:31.195131 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:31.195079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:31.295374 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:31.295338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:32.883003 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:32.882956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:38.774739 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:38.774705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:38.874333 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:38.874294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:39.319288 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:39.319249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:45.064029 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.063985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:45.163680 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.163647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:45.300255 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.300224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:45.566133 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.566100 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:05:45.566961 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.566936 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:05:45.566961 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.566959 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:05:45.567135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.566989 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40f20a28-8f9b-466b-9cf2-f50ca66da601" containerName="istio-proxy" Jul 10 21:05:45.567135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.566995 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="40f20a28-8f9b-466b-9cf2-f50ca66da601" containerName="istio-proxy" Jul 10 21:05:45.567135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.567058 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="40f20a28-8f9b-466b-9cf2-f50ca66da601" containerName="istio-proxy" Jul 10 21:05:45.567135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.567068 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:05:45.570013 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.569997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.572575 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.572555 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-76673b9a-openshift-default-dockercfg-zv8hn\"" Jul 10 21:05:45.583039 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.583013 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:45.595447 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595568 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595568 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595568 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpr6j\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595721 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.595837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.595742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.598653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.598630 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:05:45.696895 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.696861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.696902 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.696931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.696949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.696968 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697394 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697394 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpr6j\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697394 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697345 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697729 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697708 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697850 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697818 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.697902 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.697862 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.698259 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.698235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.699266 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.699246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.699820 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.699788 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.709063 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.709039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpr6j\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.711417 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.711398 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token\") pod \"e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.880366 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.880285 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:45.881883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:45.881850 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:46.027485 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.027452 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:05:46.030376 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:05:46.030349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4623cc15_8177_4b48_87a0_3a10ece89c92.slice/crio-4443aa184ae50de77ce1b14289e1b079bc5fa44958b594bb63a423fad73b34c9 WatchSource:0}: Error finding container 4443aa184ae50de77ce1b14289e1b079bc5fa44958b594bb63a423fad73b34c9: Status 404 returned error can't find the container with id 4443aa184ae50de77ce1b14289e1b079bc5fa44958b594bb63a423fad73b34c9 Jul 10 21:05:46.032497 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.032470 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:05:46.032559 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.032535 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:05:46.032597 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.032562 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:05:46.842388 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.842331 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerStarted","Data":"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7"} Jul 10 21:05:46.842775 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.842432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerStarted","Data":"4443aa184ae50de77ce1b14289e1b079bc5fa44958b594bb63a423fad73b34c9"} Jul 10 21:05:46.864622 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.864563 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" podStartSLOduration=1.8645439160000001 podStartE2EDuration="1.864543916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:05:46.861228284 +0000 UTC m=+1855.745059674" watchObservedRunningTime="2026-07-10 21:05:46.864543916 +0000 UTC m=+1855.748375261" Jul 10 21:05:46.880384 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:46.880352 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:47.885897 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:47.885869 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:48.822289 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:48.822256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:48.852462 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:48.852434 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:48.853483 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:48.853465 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:05:48.916328 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:48.916285 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:49.330495 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:49.330461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:49.436270 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:49.436223 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:49.653574 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:49.653455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:53.246930 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:53.246897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:53.930643 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:53.930610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:54.021175 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:54.021138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:05:54.340646 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:05:54.340607 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:00.132766 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:00.132734 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:00.252211 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:00.252179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:00.549776 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:00.549738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:00.960655 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:00.960566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:01.059593 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:01.059554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:03.854130 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.854095 2566 patch_prober.go:28] interesting pod/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 10 21:06:03.854547 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.854152 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 10 21:06:03.941953 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.941915 2566 generic.go:356] "Generic (PLEG): container finished" podID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerID="6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7" exitCode=0 Jul 10 21:06:03.942116 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.941990 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerDied","Data":"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7"} Jul 10 21:06:03.942495 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.942480 2566 scope.go:117] "RemoveContainer" containerID="6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7" Jul 10 21:06:03.943593 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.943568 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:03.943662 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.943643 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:03.943711 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:03.943674 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:04.948984 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:04.948949 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerStarted","Data":"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f"} Jul 10 21:06:05.881907 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:05.881173 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:06:05.881907 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:05.881566 2566 patch_prober.go:28] interesting pod/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 10 21:06:05.881907 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:05.881616 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 10 21:06:06.874702 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:06.874669 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:06.880899 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:06.880864 2566 patch_prober.go:28] interesting pod/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 10 21:06:06.881069 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:06.880927 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 10 21:06:06.976477 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:06.976406 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:07.174186 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.174099 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:06:07.174933 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.174644 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" containerID="cri-o://e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f" gracePeriod=30 Jul 10 21:06:07.208202 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.208175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:07.395157 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.395118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:07.512232 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.512201 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:07.613972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.613942 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:07.726933 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:07.726899 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:13.580717 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:13.580636 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:13.636357 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:13.636326 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:13.719713 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:13.719676 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:13.818041 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:13.818008 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:13.937400 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:13.937317 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:14.141788 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:06:14.141762 2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4623cc15_8177_4b48_87a0_3a10ece89c92.slice/crio-e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f.scope\": RecentStats: unable to find data in memory cache]" Jul 10 21:06:14.323742 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.323721 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:06:14.501455 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501424 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501478 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501515 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501543 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501566 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501732 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501751 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:06:14.501883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501784 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501857 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.501883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501869 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:06:14.502151 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501928 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpr6j\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j\") pod \"4623cc15-8177-4b48-87a0-3a10ece89c92\" (UID: \"4623cc15-8177-4b48-87a0-3a10ece89c92\") " Jul 10 21:06:14.502151 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.501972 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:06:14.502151 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502076 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:06:14.502364 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502345 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.502419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502371 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.502419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502385 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4623cc15-8177-4b48-87a0-3a10ece89c92-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.502419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502398 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.502651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.502630 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data" (OuterVolumeSpecName: "istio-data") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:06:14.503766 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.503740 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:06:14.504025 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.504002 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token" (OuterVolumeSpecName: "istio-token") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:06:14.504304 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.504283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:06:14.504352 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.504298 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j" (OuterVolumeSpecName: "kube-api-access-hpr6j") pod "4623cc15-8177-4b48-87a0-3a10ece89c92" (UID: "4623cc15-8177-4b48-87a0-3a10ece89c92"). InnerVolumeSpecName "kube-api-access-hpr6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:06:14.603791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.603757 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.603791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.603785 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.603791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.603793 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.603791 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.603801 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpr6j\" (UniqueName: \"kubernetes.io/projected/4623cc15-8177-4b48-87a0-3a10ece89c92-kube-api-access-hpr6j\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:14.604264 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:14.603827 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4623cc15-8177-4b48-87a0-3a10ece89c92-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:06:15.009515 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.009481 2566 generic.go:356] "Generic (PLEG): container finished" podID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerID="e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f" exitCode=0 Jul 10 21:06:15.009705 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.009524 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" Jul 10 21:06:15.009705 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.009563 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerDied","Data":"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f"} Jul 10 21:06:15.009705 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.009626 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l" event={"ID":"4623cc15-8177-4b48-87a0-3a10ece89c92","Type":"ContainerDied","Data":"4443aa184ae50de77ce1b14289e1b079bc5fa44958b594bb63a423fad73b34c9"} Jul 10 21:06:15.009705 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.009656 2566 scope.go:117] "RemoveContainer" containerID="e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f" Jul 10 21:06:15.021641 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.021624 2566 scope.go:117] "RemoveContainer" containerID="6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7" Jul 10 21:06:15.030027 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.030009 2566 scope.go:117] "RemoveContainer" containerID="e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f" Jul 10 21:06:15.030261 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:06:15.030237 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f\": container with ID starting with e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f not found: ID does not exist" containerID="e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f" Jul 10 21:06:15.030316 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.030271 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f"} err="failed to get container status \"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f\": rpc error: code = NotFound desc = could not find container \"e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f\": container with ID starting with e9e236e15ed7099bc8ffa8d6f7972ccaffeec6118b687d684b51f3b67eed632f not found: ID does not exist" Jul 10 21:06:15.030316 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.030288 2566 scope.go:117] "RemoveContainer" containerID="6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7" Jul 10 21:06:15.030514 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:06:15.030493 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7\": container with ID starting with 6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7 not found: ID does not exist" containerID="6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7" Jul 10 21:06:15.030556 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.030523 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7"} err="failed to get container status \"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7\": rpc error: code = NotFound desc = could not find container \"6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7\": container with ID starting with 6383a40a400acf5e61854f6eedade914ad774411ac46d88467c496659bf0aff7 not found: ID does not exist" Jul 10 21:06:15.034422 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.034398 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:06:15.039065 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.039037 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-76673b9a-openshift-default-5b5f4b7ff9-wgx5l"] Jul 10 21:06:15.597420 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:15.597386 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" path="/var/lib/kubelet/pods/4623cc15-8177-4b48-87a0-3a10ece89c92/volumes" Jul 10 21:06:19.894643 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:19.894599 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:19.994429 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:19.994398 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:26.611095 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.611062 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:26.859113 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859084 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:06:26.859620 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859605 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:06:26.859674 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859621 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:06:26.859674 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859646 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:06:26.859674 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859653 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:06:26.859837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859675 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:06:26.859837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859682 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:06:26.859837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859761 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32af0e78-25ed-4747-955c-f891d2558d84" containerName="istio-proxy" Jul 10 21:06:26.859837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.859775 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:06:26.861989 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.861940 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.864313 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.864291 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8d3029d7-openshift-default-dockercfg-sdjdh\"" Jul 10 21:06:26.875041 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.875021 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:06:26.918672 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918844 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918844 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918844 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hmkj\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918844 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918991 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918991 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.918991 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.918956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:26.919345 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:26.919001 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020407 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020595 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020419 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hmkj\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020595 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020595 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020769 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020769 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020769 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020952 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.020952 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.021107 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.021084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.021174 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.020798 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.021235 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.021199 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.021286 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.021268 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.021359 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.021342 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.022898 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.022879 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.023098 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.023080 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.028215 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.028196 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.028593 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.028572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hmkj\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj\") pod \"e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.174749 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.174668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:27.316428 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.316404 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:06:27.318793 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:06:27.318767 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25fb4c5_2c5d_4988_8a4e_04fe3ce972a8.slice/crio-29479d442ac031b855f655ec21cde4dad17877df67c828116ebd5f190a6afc07 WatchSource:0}: Error finding container 29479d442ac031b855f655ec21cde4dad17877df67c828116ebd5f190a6afc07: Status 404 returned error can't find the container with id 29479d442ac031b855f655ec21cde4dad17877df67c828116ebd5f190a6afc07 Jul 10 21:06:27.321136 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.321091 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:27.321228 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.321195 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:27.321269 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:27.321239 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:28.082489 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.082448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerStarted","Data":"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9"} Jul 10 21:06:28.082925 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.082499 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerStarted","Data":"29479d442ac031b855f655ec21cde4dad17877df67c828116ebd5f190a6afc07"} Jul 10 21:06:28.104841 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.104781 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podStartSLOduration=2.10476647 podStartE2EDuration="2.10476647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:06:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:06:28.101732228 +0000 UTC m=+1896.985563574" watchObservedRunningTime="2026-07-10 21:06:28.10476647 +0000 UTC m=+1896.988597813" Jul 10 21:06:28.175201 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.175165 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:28.180505 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.180474 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:28.835112 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:28.835079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:29.088153 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:29.088073 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:29.088963 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:29.088936 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:30.068132 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:30.068093 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:30.149057 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:30.149016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:30.297873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:30.297839 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:30.387458 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:30.387380 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:31.188636 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:31.188603 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:31.286224 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:31.286194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:34.456731 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:34.456702 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:36.399310 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:36.399277 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:36.497759 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:36.497723 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:44.089544 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.089508 2566 patch_prober.go:28] interesting pod/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 21:06:44.089966 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.089577 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 21:06:44.178213 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.178127 2566 generic.go:356] "Generic (PLEG): container finished" podID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerID="9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9" exitCode=0 Jul 10 21:06:44.178213 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.178194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerDied","Data":"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9"} Jul 10 21:06:44.178666 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.178653 2566 scope.go:117] "RemoveContainer" containerID="9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9" Jul 10 21:06:44.179860 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.179833 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:44.179960 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.179899 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:44.179960 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:44.179931 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:06:45.189915 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:45.189884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerStarted","Data":"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f"} Jul 10 21:06:46.175014 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:46.174956 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:46.175400 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:46.175365 2566 patch_prober.go:28] interesting pod/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 21:06:46.175513 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:46.175442 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 21:06:47.175711 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:47.175674 2566 patch_prober.go:28] interesting pod/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 21:06:47.176515 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:47.176471 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 21:06:48.176191 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:48.176143 2566 patch_prober.go:28] interesting pod/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 21:06:48.176695 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:48.176216 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 21:06:49.179016 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:49.178981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:49.179398 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:49.179225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:49.179985 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:49.179965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:06:51.471958 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:51.471924 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:51.572956 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:51.572924 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:06:55.647721 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:06:55.647680 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:17.057147 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:07:17.057111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:17.415621 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:07:17.415543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:23.116514 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:07:23.116482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:07:28.194369 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:07:28.194337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:03.636874 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:03.636743 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:04.497738 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:04.497702 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:05.157605 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:05.157575 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:10.807416 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:10.807381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:10.876664 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:10.876619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:11.076032 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:11.075948 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:08:11.076195 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:11.076171 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" containerID="cri-o://5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f" gracePeriod=30 Jul 10 21:08:11.096894 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:11.096862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:11.341558 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:11.341474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:12.424247 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:12.424207 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:12.526012 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:12.525979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:12.628341 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:12.628306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:13.379940 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:13.379909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:13.478439 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:13.478402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:13.578199 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:13.578168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:13.951176 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:13.951141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:15.010341 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:15.010307 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:16.346125 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.346052 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:08:16.420664 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420627 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.420848 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420683 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.420848 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420725 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.420848 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420762 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.420848 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420789 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.421082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420854 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.421082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420955 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.421082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.420983 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hmkj\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.421082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421025 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo\") pod \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\" (UID: \"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8\") " Jul 10 21:08:16.421082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421048 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:16.421320 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421116 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:16.421320 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421127 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:16.421422 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421389 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.421422 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421408 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.421511 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421423 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.421511 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421423 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:08:16.421669 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.421645 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:16.423279 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.423254 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:08:16.423610 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.423582 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:08:16.423715 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.423658 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:08:16.423715 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.423686 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj" (OuterVolumeSpecName: "kube-api-access-8hmkj") pod "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" (UID: "b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8"). InnerVolumeSpecName "kube-api-access-8hmkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:08:16.522154 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522118 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.522154 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522151 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hmkj\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-kube-api-access-8hmkj\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.522154 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522161 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.522388 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522170 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.522388 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522179 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.522388 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.522186 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:08:16.685860 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.685760 2566 generic.go:356] "Generic (PLEG): container finished" podID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerID="5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f" exitCode=0 Jul 10 21:08:16.685860 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.685827 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" Jul 10 21:08:16.686107 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.685846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerDied","Data":"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f"} Jul 10 21:08:16.686107 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.685914 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq" event={"ID":"b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8","Type":"ContainerDied","Data":"29479d442ac031b855f655ec21cde4dad17877df67c828116ebd5f190a6afc07"} Jul 10 21:08:16.686107 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.685948 2566 scope.go:117] "RemoveContainer" containerID="5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f" Jul 10 21:08:16.696590 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.696563 2566 scope.go:117] "RemoveContainer" containerID="9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9" Jul 10 21:08:16.705376 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.705353 2566 scope.go:117] "RemoveContainer" containerID="5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f" Jul 10 21:08:16.705670 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:08:16.705648 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f\": container with ID starting with 5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f not found: ID does not exist" containerID="5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f" Jul 10 21:08:16.705735 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.705681 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f"} err="failed to get container status \"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f\": rpc error: code = NotFound desc = could not find container \"5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f\": container with ID starting with 5064320d7b0841c309fe83d399218a2a4ffa5ccadbbcf055ebe45d8ae46a0c0f not found: ID does not exist" Jul 10 21:08:16.705735 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.705701 2566 scope.go:117] "RemoveContainer" containerID="9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9" Jul 10 21:08:16.705986 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:08:16.705971 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9\": container with ID starting with 9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9 not found: ID does not exist" containerID="9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9" Jul 10 21:08:16.706040 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.705992 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9"} err="failed to get container status \"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9\": rpc error: code = NotFound desc = could not find container \"9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9\": container with ID starting with 9c6ef5869383521d0633ccea8a96bfc573e8539e52e14046a0e06ec8f4ff57e9 not found: ID does not exist" Jul 10 21:08:16.711062 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.711036 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:08:16.714567 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:16.714543 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8d3029d7-openshift-default-59f7fb5555-vrmkq"] Jul 10 21:08:17.598185 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:17.598138 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" path="/var/lib/kubelet/pods/b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8/volumes" Jul 10 21:08:20.962281 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:20.962248 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:21.062407 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:21.062372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:21.651213 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:21.651177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:27.599002 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:27.598957 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:27.768168 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:27.768136 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:27.867863 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:27.867753 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:38.330935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:38.330897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:38.699985 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:38.699907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:08:59.681609 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:08:59.681570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:05.486937 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:05.486884 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:05.587387 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:05.587316 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:05.842673 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:05.842590 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:06.119625 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.119545 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:09:06.120022 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120008 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.120074 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120025 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.120074 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120034 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.120074 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120041 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.120167 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120103 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4623cc15-8177-4b48-87a0-3a10ece89c92" containerName="istio-proxy" Jul 10 21:09:06.120167 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120111 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.120167 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.120118 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25fb4c5-2c5d-4988-8a4e-04fe3ce972a8" containerName="istio-proxy" Jul 10 21:09:06.123305 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.123289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.125715 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.125697 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7b3231-openshift-default-dockercfg-zqdr5\"" Jul 10 21:09:06.137131 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.137109 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:09:06.150974 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.150947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:06.192255 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192457 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192457 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j6g5\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192457 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192333 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192457 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192675 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192675 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192529 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192675 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.192675 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.192604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294110 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294110 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294119 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294381 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294381 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294400 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j6g5\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294525 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294556 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294659 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294859 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.294957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.294883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.295272 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.295253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.296615 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.296584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.296832 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.296796 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.303109 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.303088 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.303328 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.303307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j6g5\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5\") pod \"e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.335491 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.335462 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:06.433786 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.433691 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:06.579086 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.579063 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:09:06.581726 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:09:06.581699 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480d714a_ab29_4c40_a672_e4782b0b4e82.slice/crio-8221404ba5ba88a94cbb602b869d13f51c55812b4c71edd4c6fafa69b90cc50e WatchSource:0}: Error finding container 8221404ba5ba88a94cbb602b869d13f51c55812b4c71edd4c6fafa69b90cc50e: Status 404 returned error can't find the container with id 8221404ba5ba88a94cbb602b869d13f51c55812b4c71edd4c6fafa69b90cc50e Jul 10 21:09:06.584249 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.584217 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:06.584328 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.584284 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:06.584328 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.584313 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:06.960775 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.960736 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerStarted","Data":"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f"} Jul 10 21:09:06.961005 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.960789 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerStarted","Data":"8221404ba5ba88a94cbb602b869d13f51c55812b4c71edd4c6fafa69b90cc50e"} Jul 10 21:09:06.982148 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:06.982097 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podStartSLOduration=0.982080051 podStartE2EDuration="982.080051ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:09:06.979590274 +0000 UTC m=+2055.863421617" watchObservedRunningTime="2026-07-10 21:09:06.982080051 +0000 UTC m=+2055.865911394" Jul 10 21:09:07.433827 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:07.433785 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:07.439146 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:07.439120 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:07.975251 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:07.969335 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:07.975973 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:07.975947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:08.916338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:08.916302 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:09.015136 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:09.015098 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:09.122178 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:09.122150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:09.720124 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:09.720088 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:09.820169 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:09.820136 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:13.403848 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:13.403735 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:13.738750 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:13.738722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:14.023278 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.023178 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:09:14.028672 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.028650 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.032087 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.032059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-fc7575-openshift-default-dockercfg-zbvtj\"" Jul 10 21:09:14.039899 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.039876 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:09:14.051927 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.051903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:14.188653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.188873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.188873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.188873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188827 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.188873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57p8w\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.189135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.188978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.189135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.189042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.189135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.189072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.189135 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.189105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.205905 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.205850 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:14.290047 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.289944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290389 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57p8w\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290472 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290535 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290535 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290759 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.290759 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.290728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.291211 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.291185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.292901 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.292400 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.292901 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.292675 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.292901 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.292854 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.293201 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.293176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.294747 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.294704 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.295281 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.295260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.298681 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.298626 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.300554 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.300512 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57p8w\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w\") pod \"e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.341681 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.341646 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:14.546084 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.546059 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:09:14.547595 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:09:14.547566 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfbe010c_b69f_497e_976c_80ae2a52412f.slice/crio-11f72c6f13fa9897ac6a58686da83feaf4e399aa8e071890afcb6fa72febb2ec WatchSource:0}: Error finding container 11f72c6f13fa9897ac6a58686da83feaf4e399aa8e071890afcb6fa72febb2ec: Status 404 returned error can't find the container with id 11f72c6f13fa9897ac6a58686da83feaf4e399aa8e071890afcb6fa72febb2ec Jul 10 21:09:14.549639 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.549612 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:14.549743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.549672 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:14.549743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:14.549701 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:15.009265 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.009225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerStarted","Data":"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058"} Jul 10 21:09:15.009439 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.009274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerStarted","Data":"11f72c6f13fa9897ac6a58686da83feaf4e399aa8e071890afcb6fa72febb2ec"} Jul 10 21:09:15.033478 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.033431 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podStartSLOduration=1.03341512 podStartE2EDuration="1.03341512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:09:15.031537212 +0000 UTC m=+2063.915368555" watchObservedRunningTime="2026-07-10 21:09:15.03341512 +0000 UTC m=+2063.917246463" Jul 10 21:09:15.342757 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.342664 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:15.344463 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.344427 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:09:15.344628 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:15.344494 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:09:16.346723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:16.346694 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:16.991249 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:16.991186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:17.020464 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.020439 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:17.021487 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.021466 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:17.093603 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.093562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:17.175838 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.175782 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:17.325583 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.325549 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:17.745971 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.745940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:17.846663 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:17.846632 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:21.046267 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.046237 2566 generic.go:356] "Generic (PLEG): container finished" podID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerID="02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f" exitCode=0 Jul 10 21:09:21.046696 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.046294 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerDied","Data":"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f"} Jul 10 21:09:21.046790 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.046772 2566 scope.go:117] "RemoveContainer" containerID="02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f" Jul 10 21:09:21.047941 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.047911 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:21.048050 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.048016 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:21.048118 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.048064 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:21.498494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:21.498464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:22.054139 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:22.054105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerStarted","Data":"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c"} Jul 10 21:09:22.434743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:22.434652 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:22.435737 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:22.435708 2566 patch_prober.go:28] interesting pod/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 21:09:22.435896 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:22.435763 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 21:09:22.971841 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:22.969919 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:23.434962 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:23.434926 2566 patch_prober.go:28] interesting pod/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 21:09:23.435351 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:23.435011 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 21:09:24.435085 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:24.435050 2566 patch_prober.go:28] interesting pod/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 21:09:24.435518 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:24.435103 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 21:09:25.434337 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:25.434297 2566 patch_prober.go:28] interesting pod/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 21:09:25.434518 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:25.434373 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 21:09:26.439501 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:26.439470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:26.440560 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:26.440537 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:09:30.049442 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.049400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:30.107121 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.107089 2566 generic.go:356] "Generic (PLEG): container finished" podID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerID="96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058" exitCode=0 Jul 10 21:09:30.107287 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.107162 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerDied","Data":"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058"} Jul 10 21:09:30.107603 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.107588 2566 scope.go:117] "RemoveContainer" containerID="96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058" Jul 10 21:09:30.108640 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.108615 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:30.108706 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.108681 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:30.108743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.108733 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:09:30.241494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.241460 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:30.346624 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:30.346529 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:31.114171 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:31.114139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerStarted","Data":"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974"} Jul 10 21:09:31.341999 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:31.341965 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:31.343192 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:31.343160 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:09:31.343332 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:31.343213 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:09:32.020638 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:32.020602 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:32.342119 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:32.342037 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:09:32.342119 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:32.342094 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:09:33.342795 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:33.342759 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:09:33.343197 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:33.342844 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:09:34.342301 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:34.342263 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:09:34.342498 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:34.342318 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:09:35.345998 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:35.345968 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:35.346821 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:35.346792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:09:38.582343 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:38.582301 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:38.690144 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:38.690104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:46.046342 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:46.046287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:09:51.691902 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:51.691867 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:09:51.695747 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:51.695728 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:09:51.700402 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:51.700384 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:09:51.704372 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:09:51.704355 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:10:09.654803 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:09.654761 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:10.579314 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:10.579280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:10.748763 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:10.748732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:11.532391 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:11.532356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:11.592058 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:11.592014 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:16.270660 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:16.270620 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:52.835775 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:52.835698 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:53.103740 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.103650 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:10:53.103995 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.103966 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" containerID="cri-o://68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c" gracePeriod=30 Jul 10 21:10:53.118889 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.118859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:53.360990 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.360907 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:53.548626 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.548593 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:53.976922 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:53.976889 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:54.248022 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:54.247982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:54.345798 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:54.345766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:54.549085 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:54.549006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:56.442282 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:56.442246 2566 patch_prober.go:28] interesting pod/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 21:10:56.442683 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:56.442321 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 21:10:58.361955 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.361930 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:10:58.474090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474003 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474034 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474058 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474101 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474122 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474159 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474210 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474313 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474334 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j6g5\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474352 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:58.474390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474389 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token\") pod \"480d714a-ab29-4c40-a672-e4782b0b4e82\" (UID: \"480d714a-ab29-4c40-a672-e4782b0b4e82\") " Jul 10 21:10:58.474857 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474459 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:58.474857 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474548 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:10:58.474987 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474902 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.474987 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474926 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.474987 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474940 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.474987 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.474952 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/480d714a-ab29-4c40-a672-e4782b0b4e82-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.475503 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.475475 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data" (OuterVolumeSpecName: "istio-data") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:58.476516 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.476491 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token" (OuterVolumeSpecName: "istio-token") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:10:58.476617 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.476525 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5" (OuterVolumeSpecName: "kube-api-access-6j6g5") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "kube-api-access-6j6g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:10:58.476617 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.476550 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:10:58.476617 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.476584 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "480d714a-ab29-4c40-a672-e4782b0b4e82" (UID: "480d714a-ab29-4c40-a672-e4782b0b4e82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:10:58.576437 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.576401 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.576437 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.576430 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.576437 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.576439 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j6g5\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-kube-api-access-6j6g5\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.576655 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.576448 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.576655 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.576458 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/480d714a-ab29-4c40-a672-e4782b0b4e82-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:10:58.596185 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.596159 2566 generic.go:356] "Generic (PLEG): container finished" podID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerID="68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c" exitCode=0 Jul 10 21:10:58.596307 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.596208 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" Jul 10 21:10:58.596307 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.596207 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerDied","Data":"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c"} Jul 10 21:10:58.596380 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.596333 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99" event={"ID":"480d714a-ab29-4c40-a672-e4782b0b4e82","Type":"ContainerDied","Data":"8221404ba5ba88a94cbb602b869d13f51c55812b4c71edd4c6fafa69b90cc50e"} Jul 10 21:10:58.596380 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.596364 2566 scope.go:117] "RemoveContainer" containerID="68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c" Jul 10 21:10:58.606097 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.606080 2566 scope.go:117] "RemoveContainer" containerID="02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f" Jul 10 21:10:58.614215 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.614199 2566 scope.go:117] "RemoveContainer" containerID="68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c" Jul 10 21:10:58.614468 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:10:58.614448 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c\": container with ID starting with 68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c not found: ID does not exist" containerID="68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c" Jul 10 21:10:58.614551 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.614481 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c"} err="failed to get container status \"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c\": rpc error: code = NotFound desc = could not find container \"68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c\": container with ID starting with 68c04728c47a4ee1b0dade24f021c383fcd2053d78799a0032029e22d8ac510c not found: ID does not exist" Jul 10 21:10:58.614551 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.614505 2566 scope.go:117] "RemoveContainer" containerID="02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f" Jul 10 21:10:58.614765 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:10:58.614739 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f\": container with ID starting with 02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f not found: ID does not exist" containerID="02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f" Jul 10 21:10:58.614885 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.614767 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f"} err="failed to get container status \"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f\": rpc error: code = NotFound desc = could not find container \"02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f\": container with ID starting with 02a22dd3b76126fc713a19cae7a3a2990e5a6bcc15d0bf9e30b3ea62a004ca6f not found: ID does not exist" Jul 10 21:10:58.617849 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.617828 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:10:58.621671 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:58.621651 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7b3231-openshift-default-75cd76d7b5-b4k99"] Jul 10 21:10:59.598301 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:59.598266 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" path="/var/lib/kubelet/pods/480d714a-ab29-4c40-a672-e4782b0b4e82/volumes" Jul 10 21:10:59.894022 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:59.893911 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:10:59.996042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:10:59.995990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:00.215176 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:00.215102 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:11:00.215322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:00.215302 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" containerID="cri-o://1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974" gracePeriod=30 Jul 10 21:11:00.236759 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:00.236729 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:00.450520 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:00.450483 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:00.796451 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:00.796396 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:01.087475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.087403 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:01.087875 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.087860 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:01.087931 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.087876 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:01.087968 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.087958 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:01.088002 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.087969 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:01.092459 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.092439 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.094725 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.094703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8b36b2-openshift-default-dockercfg-b27bg\"" Jul 10 21:11:01.102119 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.102095 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:01.110444 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.110422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:01.204202 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204326 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncjz2\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.204541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.204510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305402 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305402 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305402 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305645 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncjz2\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305797 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.305957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.306172 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.305975 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.306254 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.306227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.306421 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.306399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.308520 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.308493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.308834 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.308793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.314140 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.314111 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.314221 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.314186 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncjz2\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2\") pod \"e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.404454 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.404363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:01.551041 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.551009 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:01.552149 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:11:01.552121 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc5c3ed_0f57_40d3_9680_a325e0dedccf.slice/crio-e5048c8ce0ba50459bbe7f44f9407a4dce193818ddaac723ceb515a3a4470dd1 WatchSource:0}: Error finding container e5048c8ce0ba50459bbe7f44f9407a4dce193818ddaac723ceb515a3a4470dd1: Status 404 returned error can't find the container with id e5048c8ce0ba50459bbe7f44f9407a4dce193818ddaac723ceb515a3a4470dd1 Jul 10 21:11:01.554353 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.554313 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:01.554436 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.554405 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:01.554476 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.554447 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:01.623781 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.623746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerStarted","Data":"e5048c8ce0ba50459bbe7f44f9407a4dce193818ddaac723ceb515a3a4470dd1"} Jul 10 21:11:01.768522 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:01.768485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:02.630131 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:02.630095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerStarted","Data":"13ea9f2484c292d37ca87396dce8eef4d215e269fac4df2ef123b70bfcca7b7e"} Jul 10 21:11:02.651839 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:02.651770 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podStartSLOduration=1.651753995 podStartE2EDuration="1.651753995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:02.649004239 +0000 UTC m=+2171.532835598" watchObservedRunningTime="2026-07-10 21:11:02.651753995 +0000 UTC m=+2171.535585338" Jul 10 21:11:03.405197 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:03.405153 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:03.410106 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:03.410080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:03.636115 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:03.636075 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:03.636968 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:03.636949 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:04.576024 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:04.575989 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:04.674925 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:04.674893 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:04.754179 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:04.754144 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:04.852247 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:04.852158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:05.130671 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.130585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:05.238188 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.238158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:05.348547 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.348504 2566 patch_prober.go:28] interesting pod/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 21:11:05.348754 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.348581 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 21:11:05.476222 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.476197 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:11:05.548744 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548717 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.548938 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548776 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57p8w\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.548938 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548873 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.548938 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548910 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.548938 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548931 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.549127 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548957 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.549127 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.548977 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.549127 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549040 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.549127 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549062 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy\") pod \"bfbe010c-b69f-497e-976c-80ae2a52412f\" (UID: \"bfbe010c-b69f-497e-976c-80ae2a52412f\") " Jul 10 21:11:05.549335 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549106 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:05.549335 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549203 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:05.549335 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549250 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:05.549502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549481 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.549561 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549508 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.549561 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549523 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.549688 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549663 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:11:05.549951 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.549924 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data" (OuterVolumeSpecName: "istio-data") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:05.551234 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.551205 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w" (OuterVolumeSpecName: "kube-api-access-57p8w") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "kube-api-access-57p8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:05.551234 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.551218 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token" (OuterVolumeSpecName: "istio-token") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:05.551366 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.551248 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:05.551366 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.551260 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bfbe010c-b69f-497e-976c-80ae2a52412f" (UID: "bfbe010c-b69f-497e-976c-80ae2a52412f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:11:05.649925 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.649835 2566 generic.go:356] "Generic (PLEG): container finished" podID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerID="1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974" exitCode=0 Jul 10 21:11:05.649925 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.649892 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" Jul 10 21:11:05.649925 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.649911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerDied","Data":"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974"} Jul 10 21:11:05.650189 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.649979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9" event={"ID":"bfbe010c-b69f-497e-976c-80ae2a52412f","Type":"ContainerDied","Data":"11f72c6f13fa9897ac6a58686da83feaf4e399aa8e071890afcb6fa72febb2ec"} Jul 10 21:11:05.650189 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650002 2566 scope.go:117] "RemoveContainer" containerID="1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650193 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfbe010c-b69f-497e-976c-80ae2a52412f-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650216 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650231 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650246 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650260 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.650300 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.650272 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57p8w\" (UniqueName: \"kubernetes.io/projected/bfbe010c-b69f-497e-976c-80ae2a52412f-kube-api-access-57p8w\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:05.660963 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.660937 2566 scope.go:117] "RemoveContainer" containerID="96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058" Jul 10 21:11:05.673268 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.673237 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:11:05.677793 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.677774 2566 scope.go:117] "RemoveContainer" containerID="1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974" Jul 10 21:11:05.678151 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:11:05.678123 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974\": container with ID starting with 1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974 not found: ID does not exist" containerID="1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974" Jul 10 21:11:05.678195 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.678150 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974"} err="failed to get container status \"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974\": rpc error: code = NotFound desc = could not find container \"1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974\": container with ID starting with 1891d777828048a6822e9e8a570a7ebe4b2bf0a6d71e998af8660764aaaac974 not found: ID does not exist" Jul 10 21:11:05.678195 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.678169 2566 scope.go:117] "RemoveContainer" containerID="96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058" Jul 10 21:11:05.678480 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:11:05.678461 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058\": container with ID starting with 96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058 not found: ID does not exist" containerID="96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058" Jul 10 21:11:05.678529 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.678487 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058"} err="failed to get container status \"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058\": rpc error: code = NotFound desc = could not find container \"96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058\": container with ID starting with 96f982a3f1b97390ed756531b932dc85e4705a54ab5e0c346da518ca01e4f058 not found: ID does not exist" Jul 10 21:11:05.681702 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:05.681682 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-fc7575-openshift-default-68dff6b66f-94fz9"] Jul 10 21:11:07.597665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:07.597620 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" path="/var/lib/kubelet/pods/bfbe010c-b69f-497e-976c-80ae2a52412f/volumes" Jul 10 21:11:09.048100 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.048067 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:09.502110 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.502077 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:09.773506 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.773412 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:09.774083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774022 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:09.774083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774041 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:09.774083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774053 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:09.774083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774058 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="480d714a-ab29-4c40-a672-e4782b0b4e82" containerName="istio-proxy" Jul 10 21:11:09.774376 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774135 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:09.774376 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.774145 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:09.778871 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.778851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.781058 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.781039 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0b11f5-openshift-default-dockercfg-2wrtl\"" Jul 10 21:11:09.787454 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.787425 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:09.828872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.828837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:09.896094 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896285 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896285 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896285 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896191 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896285 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896464 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896464 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896464 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.896615 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.896486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6q2m\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997653 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997752 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6q2m\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997865 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.997936 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.997935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.998322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.998170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.998322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.998311 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.998420 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.998399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.998484 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.998430 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:09.998545 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:09.998486 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.000091 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.000071 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.000403 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.000386 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.005619 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.005587 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.006243 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.006224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6q2m\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m\") pod \"e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.034171 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.034081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:10.091339 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.091303 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:10.239354 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.239325 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:10.241685 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:11:10.241636 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52525ac_0860_40ef_84b0_6400635ff748.slice/crio-0c1e0bceaa9e4d6d7197792e0869c77680d51510492e66907e38350d00f31a26 WatchSource:0}: Error finding container 0c1e0bceaa9e4d6d7197792e0869c77680d51510492e66907e38350d00f31a26: Status 404 returned error can't find the container with id 0c1e0bceaa9e4d6d7197792e0869c77680d51510492e66907e38350d00f31a26 Jul 10 21:11:10.244285 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.244252 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:10.244391 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.244348 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:10.244461 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.244418 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:10.679599 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.679508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerStarted","Data":"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb"} Jul 10 21:11:10.679599 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.679564 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerStarted","Data":"0c1e0bceaa9e4d6d7197792e0869c77680d51510492e66907e38350d00f31a26"} Jul 10 21:11:10.700824 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:10.700751 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podStartSLOduration=1.700729677 podStartE2EDuration="1.700729677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:10.698452284 +0000 UTC m=+2179.582283624" watchObservedRunningTime="2026-07-10 21:11:10.700729677 +0000 UTC m=+2179.584561021" Jul 10 21:11:11.091543 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:11.091508 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:11.096486 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:11.096461 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:11.684459 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:11.684422 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:11.685279 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:11.685257 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:12.589632 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:12.589590 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:12.718053 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:12.718020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:12.819218 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:12.819165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:12.952361 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:12.952282 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:13.247717 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:13.247681 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:13.345361 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:13.345331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:15.559752 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:15.559715 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:17.092098 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:17.092060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:19.677977 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:19.677937 2566 patch_prober.go:28] interesting pod/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 21:11:19.678423 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:19.678017 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 21:11:21.745152 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.745116 2566 generic.go:356] "Generic (PLEG): container finished" podID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerID="13ea9f2484c292d37ca87396dce8eef4d215e269fac4df2ef123b70bfcca7b7e" exitCode=0 Jul 10 21:11:21.745675 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.745171 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerDied","Data":"13ea9f2484c292d37ca87396dce8eef4d215e269fac4df2ef123b70bfcca7b7e"} Jul 10 21:11:21.745738 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.745722 2566 scope.go:117] "RemoveContainer" containerID="13ea9f2484c292d37ca87396dce8eef4d215e269fac4df2ef123b70bfcca7b7e" Jul 10 21:11:21.746891 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.746866 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:21.746968 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.746933 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:21.746968 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:21.746963 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:22.754657 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:22.754619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerStarted","Data":"5b459ffb8d40c26b0043cfdd22a8d1f1adbbbf76ffb55022138b67d03accbb79"} Jul 10 21:11:23.404616 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:23.404566 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:23.404884 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:23.404856 2566 patch_prober.go:28] interesting pod/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 21:11:23.404966 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:23.404920 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 21:11:24.405845 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:24.405780 2566 patch_prober.go:28] interesting pod/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 21:11:24.406344 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:24.405917 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 21:11:25.318833 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.317856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:25.405821 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.405763 2566 patch_prober.go:28] interesting pod/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 21:11:25.406132 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.406099 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 21:11:25.627623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.627525 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:25.723518 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.723482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:25.823345 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.823311 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:25.936885 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:25.936263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:26.405867 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:26.405822 2566 patch_prober.go:28] interesting pod/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 21:11:26.406082 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:26.405909 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 21:11:27.409607 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:27.409573 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:27.410102 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:27.410071 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:27.410998 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:27.410973 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:29.807241 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.807208 2566 generic.go:356] "Generic (PLEG): container finished" podID="c52525ac-0860-40ef-84b0-6400635ff748" containerID="f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb" exitCode=0 Jul 10 21:11:29.807660 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.807279 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerDied","Data":"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb"} Jul 10 21:11:29.807727 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.807713 2566 scope.go:117] "RemoveContainer" containerID="f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb" Jul 10 21:11:29.808696 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.808668 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:29.808824 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.808730 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:29.808824 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:29.808765 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:30.814210 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:30.814179 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerStarted","Data":"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234"} Jul 10 21:11:31.092563 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.092465 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:31.092793 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.092765 2566 patch_prober.go:28] interesting pod/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 21:11:31.092890 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.092868 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 21:11:31.414171 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.414095 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:31.641639 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.641607 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:31.641885 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:31.641858 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" containerID="cri-o://5b459ffb8d40c26b0043cfdd22a8d1f1adbbbf76ffb55022138b67d03accbb79" gracePeriod=30 Jul 10 21:11:32.091964 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:32.091929 2566 patch_prober.go:28] interesting pod/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 21:11:32.092350 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:32.091984 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 21:11:32.717090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:32.717060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:32.842947 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:32.842911 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:33.092420 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:33.092384 2566 patch_prober.go:28] interesting pod/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 21:11:33.092972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:33.092920 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 21:11:34.091776 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:34.091735 2566 patch_prober.go:28] interesting pod/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 21:11:34.091976 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:34.091858 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 21:11:35.095500 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:35.095470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:35.095991 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:35.095834 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:35.096388 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:35.096369 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:36.858577 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.858545 2566 generic.go:356] "Generic (PLEG): container finished" podID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerID="5b459ffb8d40c26b0043cfdd22a8d1f1adbbbf76ffb55022138b67d03accbb79" exitCode=0 Jul 10 21:11:36.858969 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.858622 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerDied","Data":"5b459ffb8d40c26b0043cfdd22a8d1f1adbbbf76ffb55022138b67d03accbb79"} Jul 10 21:11:36.858969 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.858689 2566 scope.go:117] "RemoveContainer" containerID="13ea9f2484c292d37ca87396dce8eef4d215e269fac4df2ef123b70bfcca7b7e" Jul 10 21:11:36.906426 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.906370 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:36.981696 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981665 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981710 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981730 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981760 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981789 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981851 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncjz2\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.981908 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.981874 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.982226 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982033 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:36.982226 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982031 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.982226 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982072 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:36.982226 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982214 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data\") pod \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\" (UID: \"3fc5c3ed-0f57-40d3-9680-a325e0dedccf\") " Jul 10 21:11:36.982439 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982315 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:11:36.982546 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982515 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:36.982924 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982839 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:36.982924 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982864 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:36.982924 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982879 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:36.982924 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.982892 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:36.983501 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.983474 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:36.984370 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.984342 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:36.984475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.984455 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2" (OuterVolumeSpecName: "kube-api-access-ncjz2") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "kube-api-access-ncjz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:36.984573 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.984559 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:36.984633 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:36.984617 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fc5c3ed-0f57-40d3-9680-a325e0dedccf" (UID: "3fc5c3ed-0f57-40d3-9680-a325e0dedccf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:11:37.084044 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.084011 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:37.084044 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.084037 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:37.084044 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.084046 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncjz2\" (UniqueName: \"kubernetes.io/projected/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-kube-api-access-ncjz2\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:37.084313 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.084057 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:37.084313 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.084066 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc5c3ed-0f57-40d3-9680-a325e0dedccf-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:37.094761 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.094732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:37.195455 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.195360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:37.865699 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.865616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" event={"ID":"3fc5c3ed-0f57-40d3-9680-a325e0dedccf","Type":"ContainerDied","Data":"e5048c8ce0ba50459bbe7f44f9407a4dce193818ddaac723ceb515a3a4470dd1"} Jul 10 21:11:37.865699 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.865675 2566 scope.go:117] "RemoveContainer" containerID="5b459ffb8d40c26b0043cfdd22a8d1f1adbbbf76ffb55022138b67d03accbb79" Jul 10 21:11:37.866202 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.865623 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj" Jul 10 21:11:37.882819 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.882784 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:37.886976 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:37.886954 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8b36b2-openshift-default-59459b97b9-b8psj"] Jul 10 21:11:38.187503 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.187425 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:38.250041 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.250009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:38.434734 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.434696 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:38.434996 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.434965 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" containerID="cri-o://332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234" gracePeriod=30 Jul 10 21:11:38.456536 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.456438 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:38.678175 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:38.678139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.002789 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.002750 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.283149 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283074 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:11:39.283512 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283498 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.283549 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283514 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.283549 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283526 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.283549 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283531 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.283549 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283547 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:39.283700 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283552 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfbe010c-b69f-497e-976c-80ae2a52412f" containerName="istio-proxy" Jul 10 21:11:39.283700 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283618 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.283700 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.283626 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" containerName="istio-proxy" Jul 10 21:11:39.287909 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.287891 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.290465 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.290444 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-01037d-openshift-default-dockercfg-phnvt\"" Jul 10 21:11:39.297493 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.297463 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:11:39.307406 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.307385 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:39.407242 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407525 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407525 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc9l4\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407525 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407620 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.407620 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.407604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508203 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508203 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508486 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508486 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508279 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508486 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508486 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc9l4\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508710 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508710 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508828 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508717 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508828 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.508828 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.508739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.509042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.509016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.509042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.509030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.509217 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.509116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.510651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.510629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.510765 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.510747 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.516134 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.516110 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.516291 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.516274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc9l4\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4\") pod \"e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.598494 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.598402 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:39.599314 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.599283 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc5c3ed-0f57-40d3-9680-a325e0dedccf" path="/var/lib/kubelet/pods/3fc5c3ed-0f57-40d3-9680-a325e0dedccf/volumes" Jul 10 21:11:39.738103 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.738077 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:11:39.740134 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:11:39.740103 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcddfce2f_25d9_4bb3_bb92_8d176b7539fd.slice/crio-6f6ef04674de162e664b9ebf86338bbe2b542bdf0ea7481dd7471b31d1f10d0a WatchSource:0}: Error finding container 6f6ef04674de162e664b9ebf86338bbe2b542bdf0ea7481dd7471b31d1f10d0a: Status 404 returned error can't find the container with id 6f6ef04674de162e664b9ebf86338bbe2b542bdf0ea7481dd7471b31d1f10d0a Jul 10 21:11:39.742204 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.742175 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:39.742295 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.742235 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:39.742295 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.742263 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:39.893172 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.893074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerStarted","Data":"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2"} Jul 10 21:11:39.893172 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.893160 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerStarted","Data":"6f6ef04674de162e664b9ebf86338bbe2b542bdf0ea7481dd7471b31d1f10d0a"} Jul 10 21:11:39.916898 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.916848 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podStartSLOduration=0.916830932 podStartE2EDuration="916.830932ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:39.914389524 +0000 UTC m=+2208.798220865" watchObservedRunningTime="2026-07-10 21:11:39.916830932 +0000 UTC m=+2208.800662273" Jul 10 21:11:39.935529 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:39.935502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:40.599450 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:40.599416 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:40.604151 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:40.604126 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:40.898847 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:40.898749 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:40.899844 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:40.899801 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:41.844883 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:41.844853 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:41.916636 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:41.916591 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:41.988469 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:41.988433 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:42.163009 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:42.162913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:42.401083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:42.401049 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:42.505920 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:42.505892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:43.700957 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.700903 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:43.853312 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853279 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853331 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853348 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853376 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853440 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853530 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853564 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6q2m\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853583 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853594 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.853665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853632 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert\") pod \"c52525ac-0860-40ef-84b0-6400635ff748\" (UID: \"c52525ac-0860-40ef-84b0-6400635ff748\") " Jul 10 21:11:43.853950 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.853673 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.854090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.854032 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.854090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.854052 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:11:43.854090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.854061 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.854303 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.854104 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.854392 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.854368 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data" (OuterVolumeSpecName: "istio-data") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.855604 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.855578 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:11:43.855795 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.855770 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token" (OuterVolumeSpecName: "istio-token") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:43.855929 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.855863 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m" (OuterVolumeSpecName: "kube-api-access-v6q2m") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "kube-api-access-v6q2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:11:43.855975 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.855934 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c52525ac-0860-40ef-84b0-6400635ff748" (UID: "c52525ac-0860-40ef-84b0-6400635ff748"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:11:43.915944 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.915910 2566 generic.go:356] "Generic (PLEG): container finished" podID="c52525ac-0860-40ef-84b0-6400635ff748" containerID="332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234" exitCode=0 Jul 10 21:11:43.916100 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.915966 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" Jul 10 21:11:43.916100 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.915995 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerDied","Data":"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234"} Jul 10 21:11:43.916100 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.916056 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw" event={"ID":"c52525ac-0860-40ef-84b0-6400635ff748","Type":"ContainerDied","Data":"0c1e0bceaa9e4d6d7197792e0869c77680d51510492e66907e38350d00f31a26"} Jul 10 21:11:43.916100 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.916080 2566 scope.go:117] "RemoveContainer" containerID="332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234" Jul 10 21:11:43.926124 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.926088 2566 scope.go:117] "RemoveContainer" containerID="f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb" Jul 10 21:11:43.936841 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.936791 2566 scope.go:117] "RemoveContainer" containerID="332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234" Jul 10 21:11:43.937416 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:11:43.937383 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234\": container with ID starting with 332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234 not found: ID does not exist" containerID="332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234" Jul 10 21:11:43.937545 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.937424 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234"} err="failed to get container status \"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234\": rpc error: code = NotFound desc = could not find container \"332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234\": container with ID starting with 332c321d9f03a4b32ea38ead3a23e287e9054fd60c425d6802d2774aeb302234 not found: ID does not exist" Jul 10 21:11:43.937545 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.937445 2566 scope.go:117] "RemoveContainer" containerID="f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb" Jul 10 21:11:43.937755 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:11:43.937731 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb\": container with ID starting with f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb not found: ID does not exist" containerID="f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb" Jul 10 21:11:43.937845 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.937765 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb"} err="failed to get container status \"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb\": rpc error: code = NotFound desc = could not find container \"f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb\": container with ID starting with f4a92bccf6fd2654c0e09b1a6fb229b6aed6510d4f02e64f8fa38ba23053e0eb not found: ID does not exist" Jul 10 21:11:43.944570 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.944548 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:43.948726 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.948705 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0b11f5-openshift-default-54c67448c-vdljw"] Jul 10 21:11:43.954837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954781 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954837 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954802 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6q2m\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-kube-api-access-v6q2m\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954839 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52525ac-0860-40ef-84b0-6400635ff748-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954848 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52525ac-0860-40ef-84b0-6400635ff748-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954856 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52525ac-0860-40ef-84b0-6400635ff748-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954865 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:43.954935 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:43.954873 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52525ac-0860-40ef-84b0-6400635ff748-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:11:45.599132 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:45.599096 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52525ac-0860-40ef-84b0-6400635ff748" path="/var/lib/kubelet/pods/c52525ac-0860-40ef-84b0-6400635ff748/volumes" Jul 10 21:11:46.353623 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.353584 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:46.673750 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.673644 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:46.939638 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.939546 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:11:46.940248 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940231 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.940322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940251 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.940322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940282 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.940322 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940292 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.940420 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940388 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.940420 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.940400 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52525ac-0860-40ef-84b0-6400635ff748" containerName="istio-proxy" Jul 10 21:11:46.946174 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.946145 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:46.948454 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.948426 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e63f46-openshift-default-dockercfg-tssbf\"" Jul 10 21:11:46.954037 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.953707 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:11:46.984831 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:46.984027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:47.095605 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095784 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095627 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095784 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095784 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phghn\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095784 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095951 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095951 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095951 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.095951 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.095922 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197230 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phghn\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197230 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197230 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197376 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197502 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197834 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197765 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.197892 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.197829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.198109 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.198064 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.198236 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.198159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.198393 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.198373 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.199762 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.199744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.199875 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.199856 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.205409 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.205383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.205840 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.205802 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phghn\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn\") pod \"e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.257578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.257537 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:47.449227 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.449200 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:11:47.450360 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:11:47.450332 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82833ba8_2c80_49c7_9e90_c79afc17a545.slice/crio-02fb9774e0c88b6bb7bc0ad62fdf40c5197d7ed396fb83f349ba3abb1ce6e769 WatchSource:0}: Error finding container 02fb9774e0c88b6bb7bc0ad62fdf40c5197d7ed396fb83f349ba3abb1ce6e769: Status 404 returned error can't find the container with id 02fb9774e0c88b6bb7bc0ad62fdf40c5197d7ed396fb83f349ba3abb1ce6e769 Jul 10 21:11:47.452548 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.452515 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:47.452651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.452578 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:47.452651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.452611 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:47.703421 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.702934 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:47.945926 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.945885 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerStarted","Data":"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41"} Jul 10 21:11:47.946101 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.945943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerStarted","Data":"02fb9774e0c88b6bb7bc0ad62fdf40c5197d7ed396fb83f349ba3abb1ce6e769"} Jul 10 21:11:47.969438 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:47.969311 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podStartSLOduration=1.969289598 podStartE2EDuration="1.969289598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 21:11:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:11:47.966092083 +0000 UTC m=+2216.849923427" watchObservedRunningTime="2026-07-10 21:11:47.969289598 +0000 UTC m=+2216.853120940" Jul 10 21:11:48.258268 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:48.258233 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:48.264012 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:48.263745 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:48.950823 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:48.950793 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:48.951922 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:48.951902 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:11:49.883662 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:49.883619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:49.982540 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:49.981841 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:50.045199 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:50.045163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:50.502725 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:50.502689 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:50.603333 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:50.603294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:50.703744 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:50.703713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:53.984694 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.984659 2566 generic.go:356] "Generic (PLEG): container finished" podID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerID="143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2" exitCode=0 Jul 10 21:11:53.985166 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.984733 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerDied","Data":"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2"} Jul 10 21:11:53.985166 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.985130 2566 scope.go:117] "RemoveContainer" containerID="143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2" Jul 10 21:11:53.986189 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.986154 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:53.986329 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.986243 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:53.986329 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:53.986297 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:11:54.443429 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:54.443393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:11:55.000401 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:55.000364 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerStarted","Data":"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622"} Jul 10 21:11:55.602125 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:55.602044 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:55.603541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:55.603515 2566 patch_prober.go:28] interesting pod/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 21:11:55.603777 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:55.603751 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 21:11:55.899768 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:55.899670 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:56.599716 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:56.599676 2566 patch_prober.go:28] interesting pod/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 21:11:56.600312 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:56.600273 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 21:11:57.599786 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:57.599740 2566 patch_prober.go:28] interesting pod/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 21:11:57.600316 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:57.599832 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 21:11:58.599042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:58.598999 2566 patch_prober.go:28] interesting pod/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 21:11:58.599228 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:58.599072 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 21:11:59.602922 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:59.602893 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:11:59.603975 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:11:59.603954 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:12:02.051656 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.051611 2566 generic.go:356] "Generic (PLEG): container finished" podID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerID="71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41" exitCode=0 Jul 10 21:12:02.052134 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.051685 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerDied","Data":"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41"} Jul 10 21:12:02.052180 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.052145 2566 scope.go:117] "RemoveContainer" containerID="71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41" Jul 10 21:12:02.053181 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.053152 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:12:02.053294 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.053234 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:12:02.053294 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:02.053280 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 21:12:03.059083 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.059046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerStarted","Data":"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17"} Jul 10 21:12:03.258241 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.258200 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:12:03.259465 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.259441 2566 patch_prober.go:28] interesting pod/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 21:12:03.259578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.259489 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 21:12:03.326190 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.326097 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:03.433226 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.433193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:03.951687 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:03.951644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:12:04.258710 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:04.258672 2566 patch_prober.go:28] interesting pod/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 21:12:04.259134 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:04.258727 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 21:12:04.470417 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:04.470371 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:05.258143 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:05.258109 2566 patch_prober.go:28] interesting pod/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 21:12:05.258324 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:05.258163 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 21:12:06.258563 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:06.258521 2566 patch_prober.go:28] interesting pod/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 21:12:06.259090 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:06.258591 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 21:12:07.262113 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:07.262080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:12:07.263208 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:07.263185 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:12:11.262112 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:11.262063 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:11.363419 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:11.363383 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:22.666944 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:22.666842 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:52.688405 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:52.688362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:58.990255 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:58.990214 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:12:59.107051 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:12:59.106993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:10.082048 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:10.081962 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:10.160529 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:10.160490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:31.941175 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:31.941128 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:32.652310 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:32.652274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:46.704600 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:46.704550 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:14:51.727475 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:51.727439 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:14:51.731540 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:51.731518 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:14:51.736912 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:51.736894 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:14:51.741019 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:14:51.741001 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2577q_cb9a1dcd-09c5-4a70-9fbf-57616552087a/ovn-acl-logging/0.log" Jul 10 21:15:08.386237 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:15:08.386182 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:32.482037 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:15:32.481994 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:32.579030 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:15:32.578997 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:52.208009 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:15:52.207972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:15:53.406001 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:15:53.405960 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:11.268469 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:11.268430 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:12.653666 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:12.653634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:24.050109 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:24.050076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:37.993023 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:37.992976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:39.234178 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:39.234139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:16:40.259122 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:16:40.259081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:04.082601 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:04.082544 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:04.182562 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:04.182503 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:12.093119 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:12.093068 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:12.196206 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:12.196172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:28.136958 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:28.136926 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:28.947559 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:28.947521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:47.800444 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:47.800405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:48.188097 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:48.188009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:48.288952 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:48.288920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:48.390943 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:48.390898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:48.516057 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:48.516023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:54.122085 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.122054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:54.188425 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.188348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:54.353273 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.353239 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:17:54.353464 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.353432 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" containerID="cri-o://cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622" gracePeriod=30 Jul 10 21:17:54.381201 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.381114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:54.609671 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.609634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:54.783849 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:54.783791 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:55.180183 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:55.180093 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:55.293299 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:55.293268 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:55.460148 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:55.460061 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:55.691350 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:55.691318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:17:59.616057 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.616009 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:17:59.667101 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667072 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667101 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667104 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667148 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667194 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc9l4\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667233 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667281 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667338 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667330 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667667 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667400 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy\") pod \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\" (UID: \"cddfce2f-25d9-4bb3-bb92-8d176b7539fd\") " Jul 10 21:17:59.667667 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667428 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:59.667667 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667580 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:59.667852 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667800 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:59.667949 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667926 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.668042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667954 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.668042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.667968 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.668042 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.668026 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:59.668469 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.668430 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:17:59.669521 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.669498 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:17:59.669852 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.669831 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:17:59.670020 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.669996 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4" (OuterVolumeSpecName: "kube-api-access-nc9l4") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "kube-api-access-nc9l4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:17:59.670150 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.670127 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "cddfce2f-25d9-4bb3-bb92-8d176b7539fd" (UID: "cddfce2f-25d9-4bb3-bb92-8d176b7539fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:17:59.768796 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768764 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.768796 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768792 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.768796 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768801 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.769025 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768837 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc9l4\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-kube-api-access-nc9l4\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.769025 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768847 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:17:59.769025 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:17:59.768856 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cddfce2f-25d9-4bb3-bb92-8d176b7539fd-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:00.016920 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.016887 2566 generic.go:356] "Generic (PLEG): container finished" podID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerID="cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622" exitCode=0 Jul 10 21:18:00.017081 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.016939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerDied","Data":"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622"} Jul 10 21:18:00.017081 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.016944 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" Jul 10 21:18:00.017081 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.016982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" event={"ID":"cddfce2f-25d9-4bb3-bb92-8d176b7539fd","Type":"ContainerDied","Data":"6f6ef04674de162e664b9ebf86338bbe2b542bdf0ea7481dd7471b31d1f10d0a"} Jul 10 21:18:00.017081 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.017003 2566 scope.go:117] "RemoveContainer" containerID="cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622" Jul 10 21:18:00.026725 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.026709 2566 scope.go:117] "RemoveContainer" containerID="143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2" Jul 10 21:18:00.035016 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.035000 2566 scope.go:117] "RemoveContainer" containerID="cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622" Jul 10 21:18:00.035278 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:18:00.035256 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622\": container with ID starting with cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622 not found: ID does not exist" containerID="cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622" Jul 10 21:18:00.035362 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.035291 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622"} err="failed to get container status \"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622\": rpc error: code = NotFound desc = could not find container \"cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622\": container with ID starting with cdc832018a717dcb74ad33a366fe096b22690f92743fe40d029f5df136558622 not found: ID does not exist" Jul 10 21:18:00.035362 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.035318 2566 scope.go:117] "RemoveContainer" containerID="143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2" Jul 10 21:18:00.035559 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:18:00.035542 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2\": container with ID starting with 143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2 not found: ID does not exist" containerID="143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2" Jul 10 21:18:00.035598 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.035565 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2"} err="failed to get container status \"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2\": rpc error: code = NotFound desc = could not find container \"143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2\": container with ID starting with 143b3d49f082d6161d16a84483d5004df0bf68d981d2b27a669de28e1f7417c2 not found: ID does not exist" Jul 10 21:18:00.039004 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.038985 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:18:00.043343 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.043321 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98"] Jul 10 21:18:00.603723 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.603685 2566 patch_prober.go:28] interesting pod/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 21:18:00.603920 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:00.603741 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-01037d-openshift-default-6cff6d59cf-sqc98" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 21:18:01.081533 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.081486 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:01.182292 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.182257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:01.341983 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.341908 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:18:01.342208 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.342181 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" containerID="cri-o://653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17" gracePeriod=30 Jul 10 21:18:01.367161 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.367127 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:01.590075 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.590042 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:01.598760 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:01.598687 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" path="/var/lib/kubelet/pods/cddfce2f-25d9-4bb3-bb92-8d176b7539fd/volumes" Jul 10 21:18:02.053238 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:02.053205 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d999c77f5-bbbtt_6723c0cf-2aac-47da-ab18-d91606325dd2/authorino/0.log" Jul 10 21:18:06.605159 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.605099 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:18:06.640597 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640564 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640597 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640594 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640612 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640652 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640723 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640773 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640796 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.640872 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640847 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phghn\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.641183 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640907 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo\") pod \"82833ba8-2c80-49c7-9e90-c79afc17a545\" (UID: \"82833ba8-2c80-49c7-9e90-c79afc17a545\") " Jul 10 21:18:06.641183 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.640915 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:18:06.641292 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.641199 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:18:06.641292 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.641203 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.641657 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.641614 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:18:06.641771 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.641626 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data" (OuterVolumeSpecName: "istio-data") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:18:06.641877 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.641825 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 21:18:06.643556 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.643523 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 21:18:06.643847 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.643795 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 21:18:06.643932 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.643838 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn" (OuterVolumeSpecName: "kube-api-access-phghn") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "kube-api-access-phghn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:18:06.644158 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.644142 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token" (OuterVolumeSpecName: "istio-token") pod "82833ba8-2c80-49c7-9e90-c79afc17a545" (UID: "82833ba8-2c80-49c7-9e90-c79afc17a545"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 21:18:06.742321 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742282 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-podinfo\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742321 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742311 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-data\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742332 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-envoy\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742345 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-workload-certs\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742356 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82833ba8-2c80-49c7-9e90-c79afc17a545-credential-socket\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742367 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-istio-token\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742378 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82833ba8-2c80-49c7-9e90-c79afc17a545-istiod-ca-cert\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.742542 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.742389 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phghn\" (UniqueName: \"kubernetes.io/projected/82833ba8-2c80-49c7-9e90-c79afc17a545-kube-api-access-phghn\") on node \"ip-10-0-137-185.ec2.internal\" DevicePath \"\"" Jul 10 21:18:06.841442 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.841393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4dpkm_7f6a6fdc-3484-4755-9ac9-f01810bcd773/manager/0.log" Jul 10 21:18:06.953287 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:06.953206 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4rtrm_bdac9a85-5cf2-427b-878f-da31494bcb49/manager/0.log" Jul 10 21:18:07.056427 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.056396 2566 generic.go:356] "Generic (PLEG): container finished" podID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerID="653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17" exitCode=0 Jul 10 21:18:07.056574 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.056453 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" Jul 10 21:18:07.056574 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.056477 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerDied","Data":"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17"} Jul 10 21:18:07.056574 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.056554 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq" event={"ID":"82833ba8-2c80-49c7-9e90-c79afc17a545","Type":"ContainerDied","Data":"02fb9774e0c88b6bb7bc0ad62fdf40c5197d7ed396fb83f349ba3abb1ce6e769"} Jul 10 21:18:07.056717 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.056578 2566 scope.go:117] "RemoveContainer" containerID="653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17" Jul 10 21:18:07.066776 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.066756 2566 scope.go:117] "RemoveContainer" containerID="71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41" Jul 10 21:18:07.076688 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.076663 2566 scope.go:117] "RemoveContainer" containerID="653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17" Jul 10 21:18:07.077016 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:18:07.076994 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17\": container with ID starting with 653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17 not found: ID does not exist" containerID="653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17" Jul 10 21:18:07.077121 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.077029 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17"} err="failed to get container status \"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17\": rpc error: code = NotFound desc = could not find container \"653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17\": container with ID starting with 653a2df2d8f12b678fdfce7792fa9e80cfa10d5a29d799482692ee02ad27be17 not found: ID does not exist" Jul 10 21:18:07.077121 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.077053 2566 scope.go:117] "RemoveContainer" containerID="71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41" Jul 10 21:18:07.077656 ip-10-0-137-185 kubenswrapper[2566]: E0710 21:18:07.077441 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41\": container with ID starting with 71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41 not found: ID does not exist" containerID="71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41" Jul 10 21:18:07.077656 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.077488 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41"} err="failed to get container status \"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41\": rpc error: code = NotFound desc = could not find container \"71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41\": container with ID starting with 71d1549eb967e420d7b7defacc1ccaf01e16aeb8d45e76a038c1bdb951f2ce41 not found: ID does not exist" Jul 10 21:18:07.081590 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.081569 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:18:07.086531 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.086507 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e63f46-openshift-default-54466f7679-wc8nq"] Jul 10 21:18:07.214705 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.214630 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-db5gc_ce32507d-7fec-4c1b-b508-42abf660f61a/server/0.log" Jul 10 21:18:07.333368 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.333333 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bkqfb_b20a0b31-9eb3-4f6a-9803-d4991413859f/manager/0.log" Jul 10 21:18:07.597973 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.597938 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" path="/var/lib/kubelet/pods/82833ba8-2c80-49c7-9e90-c79afc17a545/volumes" Jul 10 21:18:07.679476 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.679450 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5798c7df-xk6j9_c0074893-134d-41c6-81b6-090b0a80cbb2/copy-manifests/0.log" Jul 10 21:18:07.692989 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:07.692961 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5798c7df-xk6j9_c0074893-134d-41c6-81b6-090b0a80cbb2/manager/0.log" Jul 10 21:18:09.421744 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.421717 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d999c77f5-bbbtt_6723c0cf-2aac-47da-ab18-d91606325dd2/authorino/0.log" Jul 10 21:18:09.538402 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.538374 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lzwzx_7a3edc5d-f465-442a-8858-b2b76500c072/manager/0.log" Jul 10 21:18:09.649334 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.649302 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m5w5n_75a98ad8-e691-4d5a-83ce-1e9bb26da412/manager/0.log" Jul 10 21:18:09.757131 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.757100 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7m22_a798c131-c3bc-41f4-b787-619b32e3335a/kuadrant-console-plugin/0.log" Jul 10 21:18:09.876748 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.876720 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9zwg5_19d1a82a-4c4f-405d-bca4-e97df73530e7/registry-server/0.log" Jul 10 21:18:09.993699 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:09.993670 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sl2x8_f348749e-53a1-4882-bd4d-6bc6f8ae9bf4/manager/0.log" Jul 10 21:18:10.223375 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:10.223285 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-57gpf_7d0121b1-9b69-43fb-9c60-f7e5e671936b/manager/0.log" Jul 10 21:18:10.587911 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:10.587883 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47djks_e5d156fd-93b2-4e76-a510-e2e36422448d/istio-proxy/0.log" Jul 10 21:18:10.933868 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:10.933782 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d744644f5-xbcjc_04e40346-1e41-42cf-9d52-b04c72845cba/kube-auth-proxy/0.log" Jul 10 21:18:11.051076 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:11.051045 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/1.log" Jul 10 21:18:11.407652 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:11.407613 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f8879f46-24676_35d11801-b792-4e8f-b178-2c9d0c3cd49a/router/0.log" Jul 10 21:18:11.757219 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:11.757196 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-k2mwb_c73b96c1-bbaa-4b32-9656-e504c0150ada/main/0.log" Jul 10 21:18:12.214272 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:12.214192 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lvn48_39498b6f-2ba7-401d-849f-1b123571b4c5/main/0.log" Jul 10 21:18:16.002496 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:16.002402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:16.072279 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:16.072242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" limit=1024 Jul 10 21:18:18.815771 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:18.815738 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4lx9r_092b3231-427e-4ba4-99a2-ca436e68be3b/global-pull-secret-syncer/0.log" Jul 10 21:18:18.966390 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:18.966356 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2dfsf_907d43a7-938d-4b6b-bc24-27079a0ad349/konnectivity-agent/0.log" Jul 10 21:18:19.059561 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:19.059533 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-185.ec2.internal_b338db47c4937c088ae739177bb426e7/haproxy/0.log" Jul 10 21:18:24.128920 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.128886 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d999c77f5-bbbtt_6723c0cf-2aac-47da-ab18-d91606325dd2/authorino/0.log" Jul 10 21:18:24.180491 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.180464 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lzwzx_7a3edc5d-f465-442a-8858-b2b76500c072/manager/0.log" Jul 10 21:18:24.226366 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.226338 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-m5w5n_75a98ad8-e691-4d5a-83ce-1e9bb26da412/manager/0.log" Jul 10 21:18:24.275119 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.275091 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-d7m22_a798c131-c3bc-41f4-b787-619b32e3335a/kuadrant-console-plugin/0.log" Jul 10 21:18:24.338017 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.337986 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9zwg5_19d1a82a-4c4f-405d-bca4-e97df73530e7/registry-server/0.log" Jul 10 21:18:24.447711 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.447614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sl2x8_f348749e-53a1-4882-bd4d-6bc6f8ae9bf4/manager/0.log" Jul 10 21:18:24.540612 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:24.540583 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-57gpf_7d0121b1-9b69-43fb-9c60-f7e5e671936b/manager/0.log" Jul 10 21:18:26.229352 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.229327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/alertmanager/0.log" Jul 10 21:18:26.254418 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.254393 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/config-reloader/0.log" Jul 10 21:18:26.277881 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.277802 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/kube-rbac-proxy-web/0.log" Jul 10 21:18:26.300163 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.300138 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/kube-rbac-proxy/0.log" Jul 10 21:18:26.323148 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.323125 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/kube-rbac-proxy-metric/0.log" Jul 10 21:18:26.346084 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.346064 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/prom-label-proxy/0.log" Jul 10 21:18:26.368007 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.367986 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_67cd844e-96f3-4352-8975-d84f72dd484c/init-config-reloader/0.log" Jul 10 21:18:26.406036 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.406007 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-kb7cq_b64ece5b-3549-48e5-bce7-33827927b4f1/cluster-monitoring-operator/0.log" Jul 10 21:18:26.437427 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.437387 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-q4v9b_a2ef422d-131c-4153-8fb2-7eabb17baa8a/kube-state-metrics/0.log" Jul 10 21:18:26.460968 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.460948 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-q4v9b_a2ef422d-131c-4153-8fb2-7eabb17baa8a/kube-rbac-proxy-main/0.log" Jul 10 21:18:26.489078 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.489052 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-q4v9b_a2ef422d-131c-4153-8fb2-7eabb17baa8a/kube-rbac-proxy-self/0.log" Jul 10 21:18:26.519698 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.519678 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-566bf96ddb-b5v5z_28b9c04d-6d28-4785-9994-e73ff48089af/metrics-server/0.log" Jul 10 21:18:26.551473 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.551410 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-p7rvr_04d2a6b0-c511-4d27-9610-037a3df3b348/monitoring-plugin/0.log" Jul 10 21:18:26.796582 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.796556 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cv28x_0a15588c-6e7b-4456-9bf3-d11e304242f9/node-exporter/0.log" Jul 10 21:18:26.833534 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.833468 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cv28x_0a15588c-6e7b-4456-9bf3-d11e304242f9/kube-rbac-proxy/0.log" Jul 10 21:18:26.876886 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.876862 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cv28x_0a15588c-6e7b-4456-9bf3-d11e304242f9/init-textfile/0.log" Jul 10 21:18:26.989301 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:26.989274 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bqnp5_e950d224-6ecb-4395-8124-5df44622e213/kube-rbac-proxy-main/0.log" Jul 10 21:18:27.018289 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.018267 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bqnp5_e950d224-6ecb-4395-8124-5df44622e213/kube-rbac-proxy-self/0.log" Jul 10 21:18:27.042847 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.042796 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-bqnp5_e950d224-6ecb-4395-8124-5df44622e213/openshift-state-metrics/0.log" Jul 10 21:18:27.340309 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.340286 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c47c6d5b-4tbdf_ccd9cf00-e4fc-43d4-9cd7-01d0211fc362/telemeter-client/0.log" Jul 10 21:18:27.365079 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.365051 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c47c6d5b-4tbdf_ccd9cf00-e4fc-43d4-9cd7-01d0211fc362/reload/0.log" Jul 10 21:18:27.388327 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.388297 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c47c6d5b-4tbdf_ccd9cf00-e4fc-43d4-9cd7-01d0211fc362/kube-rbac-proxy/0.log" Jul 10 21:18:27.417393 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.417365 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/thanos-query/0.log" Jul 10 21:18:27.438819 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.438793 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/kube-rbac-proxy-web/0.log" Jul 10 21:18:27.467876 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.467856 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/kube-rbac-proxy/0.log" Jul 10 21:18:27.490938 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.490914 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/prom-label-proxy/0.log" Jul 10 21:18:27.513356 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.513336 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/kube-rbac-proxy-rules/0.log" Jul 10 21:18:27.537714 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:27.537654 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fb47977d8-7w7qc_8a80bd50-ef2a-47ec-a826-1f25b10da7f8/kube-rbac-proxy-metrics/0.log" Jul 10 21:18:28.803493 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803459 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2"] Jul 10 21:18:28.803912 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803898 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803912 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803924 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803929 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803939 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803945 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803956 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.803972 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.803962 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.804187 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.804030 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.804187 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.804037 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="82833ba8-2c80-49c7-9e90-c79afc17a545" containerName="istio-proxy" Jul 10 21:18:28.804187 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.804044 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.804187 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.804052 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cddfce2f-25d9-4bb3-bb92-8d176b7539fd" containerName="istio-proxy" Jul 10 21:18:28.807172 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.807154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:28.809465 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.809444 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6snqv\"/\"openshift-service-ca.crt\"" Jul 10 21:18:28.810302 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.810286 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6snqv\"/\"kube-root-ca.crt\"" Jul 10 21:18:28.810367 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.810302 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6snqv\"/\"default-dockercfg-5mrqk\"" Jul 10 21:18:28.819527 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.819504 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2"] Jul 10 21:18:28.949828 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.949777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-podres\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:28.949828 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.949826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj22v\" (UniqueName: \"kubernetes.io/projected/66cca48f-ec54-4b78-8613-431a345a49ec-kube-api-access-mj22v\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:28.950039 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.949860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-lib-modules\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:28.950039 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.949897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-sys\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:28.950039 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:28.949927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-proc\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.050873 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.050838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-podres\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051038 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.050882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj22v\" (UniqueName: \"kubernetes.io/projected/66cca48f-ec54-4b78-8613-431a345a49ec-kube-api-access-mj22v\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051038 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.050922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-lib-modules\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051038 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.050952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-sys\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051038 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.050999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-proc\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051038 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.051010 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-podres\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051209 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.051054 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-sys\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051209 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.051101 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-lib-modules\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.051209 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.051101 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66cca48f-ec54-4b78-8613-431a345a49ec-proc\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.059855 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.059772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj22v\" (UniqueName: \"kubernetes.io/projected/66cca48f-ec54-4b78-8613-431a345a49ec-kube-api-access-mj22v\") pod \"perf-node-gather-daemonset-85dn2\" (UID: \"66cca48f-ec54-4b78-8613-431a345a49ec\") " pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.117911 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.117879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:29.253907 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.253873 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2"] Jul 10 21:18:29.256968 ip-10-0-137-185 kubenswrapper[2566]: W0710 21:18:29.256940 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66cca48f_ec54_4b78_8613_431a345a49ec.slice/crio-67dedaca4e33447e19d01b58868bece5450eeda37a230fdc1436a3409b673fb2 WatchSource:0}: Error finding container 67dedaca4e33447e19d01b58868bece5450eeda37a230fdc1436a3409b673fb2: Status 404 returned error can't find the container with id 67dedaca4e33447e19d01b58868bece5450eeda37a230fdc1436a3409b673fb2 Jul 10 21:18:29.338353 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.338293 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/1.log" Jul 10 21:18:29.342627 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:29.342609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-764xs_8afb6140-2e68-4204-8c67-8f0e6c4cc4c3/console-operator/2.log" Jul 10 21:18:30.188260 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:30.188221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" event={"ID":"66cca48f-ec54-4b78-8613-431a345a49ec","Type":"ContainerStarted","Data":"32d1b5603cab669e9ef44608b637c8111de5e7e7cc5b4907c127e929f90e874c"} Jul 10 21:18:30.188665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:30.188284 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" event={"ID":"66cca48f-ec54-4b78-8613-431a345a49ec","Type":"ContainerStarted","Data":"67dedaca4e33447e19d01b58868bece5450eeda37a230fdc1436a3409b673fb2"} Jul 10 21:18:30.188665 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:30.188435 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:30.206240 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:30.206192 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" podStartSLOduration=2.206178103 podStartE2EDuration="2.206178103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 21:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 21:18:30.204213964 +0000 UTC m=+2619.088045310" watchObservedRunningTime="2026-07-10 21:18:30.206178103 +0000 UTC m=+2619.090009493" Jul 10 21:18:31.512468 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:31.512436 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xtph7_2d544536-846f-4bd5-a1af-1d9b4f8bc3a2/dns/0.log" Jul 10 21:18:31.538651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:31.538631 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xtph7_2d544536-846f-4bd5-a1af-1d9b4f8bc3a2/kube-rbac-proxy/0.log" Jul 10 21:18:31.567876 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:31.567846 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hfcmm_a341f91e-55e6-42a5-b430-2b2c3b2827e9/dns-node-resolver/0.log" Jul 10 21:18:32.135565 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:32.135536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74b7fd9d66-gsjvm_0865e613-a5bf-4383-b8cc-18ff949bdea4/registry/0.log" Jul 10 21:18:32.211555 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:32.211522 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wlsl8_1d1408fb-0169-487a-ac62-3cdf58ef7c1d/node-ca/0.log" Jul 10 21:18:33.160978 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:33.160950 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47djks_e5d156fd-93b2-4e76-a510-e2e36422448d/istio-proxy/0.log" Jul 10 21:18:33.399149 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:33.399119 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d744644f5-xbcjc_04e40346-1e41-42cf-9d52-b04c72845cba/kube-auth-proxy/0.log" Jul 10 21:18:33.431359 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:33.431277 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/0.log" Jul 10 21:18:33.461770 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:33.461743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/1.log" Jul 10 21:18:33.571745 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:33.571715 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f8879f46-24676_35d11801-b792-4e8f-b178-2c9d0c3cd49a/router/0.log" Jul 10 21:18:34.254743 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:34.254686 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2tjkp_58d1a662-8361-4825-b188-9d85c7f5ef94/serve-healthcheck-canary/0.log" Jul 10 21:18:34.919465 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:34.919434 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmrzc_278c6a6e-33a5-4918-9976-4dbe8b1223a8/kube-rbac-proxy/0.log" Jul 10 21:18:34.944298 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:34.944274 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmrzc_278c6a6e-33a5-4918-9976-4dbe8b1223a8/exporter/0.log" Jul 10 21:18:34.968673 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:34.968648 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmrzc_278c6a6e-33a5-4918-9976-4dbe8b1223a8/extractor/0.log" Jul 10 21:18:36.204406 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:36.204375 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6snqv/perf-node-gather-daemonset-85dn2" Jul 10 21:18:37.177456 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.177431 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4dpkm_7f6a6fdc-3484-4755-9ac9-f01810bcd773/manager/0.log" Jul 10 21:18:37.306279 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.306248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4rtrm_bdac9a85-5cf2-427b-878f-da31494bcb49/manager/0.log" Jul 10 21:18:37.492248 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.492219 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-db5gc_ce32507d-7fec-4c1b-b508-42abf660f61a/server/0.log" Jul 10 21:18:37.561578 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.561549 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bkqfb_b20a0b31-9eb3-4f6a-9803-d4991413859f/manager/0.log" Jul 10 21:18:37.740015 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.739984 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5798c7df-xk6j9_c0074893-134d-41c6-81b6-090b0a80cbb2/manager/0.log" Jul 10 21:18:37.758621 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:37.758557 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d5798c7df-xk6j9_c0074893-134d-41c6-81b6-090b0a80cbb2/copy-manifests/0.log" Jul 10 21:18:39.175891 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.175859 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-7g2tq_ac3052ea-2573-480e-8ce9-ef3685353b68/openshift-lws-operator/0.log" Jul 10 21:18:39.454829 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.454726 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47djks_e5d156fd-93b2-4e76-a510-e2e36422448d/istio-proxy/0.log" Jul 10 21:18:39.673735 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.673712 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d744644f5-xbcjc_04e40346-1e41-42cf-9d52-b04c72845cba/kube-auth-proxy/0.log" Jul 10 21:18:39.696698 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.696669 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/0.log" Jul 10 21:18:39.729651 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.729576 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/1.log" Jul 10 21:18:39.808349 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:39.808319 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f8879f46-24676_35d11801-b792-4e8f-b178-2c9d0c3cd49a/router/0.log" Jul 10 21:18:41.067594 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:41.067556 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47djks_e5d156fd-93b2-4e76-a510-e2e36422448d/istio-proxy/0.log" Jul 10 21:18:41.290679 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:41.290653 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d744644f5-xbcjc_04e40346-1e41-42cf-9d52-b04c72845cba/kube-auth-proxy/0.log" Jul 10 21:18:41.313939 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:41.313911 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/0.log" Jul 10 21:18:41.343298 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:41.343234 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-8629b_e28cb92f-9545-4df9-aef7-fbbc1d3642a8/istio-proxy/1.log" Jul 10 21:18:41.428798 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:41.428772 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-9f8879f46-24676_35d11801-b792-4e8f-b178-2c9d0c3cd49a/router/0.log" Jul 10 21:18:45.832314 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.832285 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/kube-multus-additional-cni-plugins/0.log" Jul 10 21:18:45.857207 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.857173 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/egress-router-binary-copy/0.log" Jul 10 21:18:45.881541 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.881519 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/cni-plugins/0.log" Jul 10 21:18:45.904638 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.904612 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/bond-cni-plugin/0.log" Jul 10 21:18:45.928242 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.928226 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/routeoverride-cni/0.log" Jul 10 21:18:45.950774 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.950751 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/whereabouts-cni-bincopy/0.log" Jul 10 21:18:45.973149 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:45.973125 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jf7ht_979fdca6-b1a1-4f8e-bdf6-17f49a4aff69/whereabouts-cni/0.log" Jul 10 21:18:46.083647 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:46.083570 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vqbfr_b54b2f36-4c49-40fe-9fcc-36c74d12a977/kube-multus/0.log" Jul 10 21:18:46.211079 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:46.211033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w5tk4_04669449-ce1e-40ad-804b-7b1bb3f744c6/network-metrics-daemon/0.log" Jul 10 21:18:46.237255 ip-10-0-137-185 kubenswrapper[2566]: I0710 21:18:46.237231 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w5tk4_04669449-ce1e-40ad-804b-7b1bb3f744c6/kube-rbac-proxy/0.log"