Jul 07 13:16:25.814198 ip-10-0-133-20 systemd[1]: Starting Kubernetes Kubelet... Jul 07 13:16:26.208744 ip-10-0-133-20 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:16:26.208744 ip-10-0-133-20 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 13:16:26.208744 ip-10-0-133-20 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:16:26.208744 ip-10-0-133-20 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 13:16:26.208744 ip-10-0-133-20 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 13:16:26.209406 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.208832 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 13:16:26.215381 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215362 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.215381 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215379 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.215381 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215383 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215386 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215389 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215393 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215395 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215398 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215400 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215403 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215406 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215408 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215411 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215413 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215417 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215419 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215427 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215430 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215433 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215436 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215438 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215441 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.215471 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215444 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215446 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215449 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215451 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215454 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215457 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215459 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215463 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215465 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215468 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215471 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215473 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215476 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215479 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215482 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215485 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215490 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215493 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215496 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215499 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.215953 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215502 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215505 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215508 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215510 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215514 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215517 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215520 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215522 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215525 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215527 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215529 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215532 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215536 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215539 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215543 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215546 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215550 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215553 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215556 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.216426 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215559 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215562 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215565 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215569 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215571 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215574 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215576 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215579 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215581 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215584 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215588 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215591 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215593 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215596 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215599 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215601 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215604 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215606 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215610 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215613 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.216878 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215615 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215618 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215620 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215623 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215625 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215628 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215631 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215634 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215636 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215639 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215642 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215644 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215647 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215649 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215652 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215655 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215658 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.215660 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216387 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216395 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.217362 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216398 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216401 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216404 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216406 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216409 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216412 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216414 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216417 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216419 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216422 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216424 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216428 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216431 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216434 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216437 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216439 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216442 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216445 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216448 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216451 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.217844 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216454 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216456 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216459 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216462 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216464 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216467 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216469 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216472 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216475 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216478 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216480 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216483 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216485 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216488 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216492 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216495 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216498 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216502 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216505 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.218345 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216508 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216510 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216514 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216517 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216520 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216523 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216526 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216529 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216531 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216534 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216536 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216539 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216542 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216544 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216547 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216549 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216552 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216554 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216558 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216562 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.218809 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216565 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216568 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216571 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216573 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216576 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216578 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216581 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216583 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216587 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216589 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216592 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216595 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216597 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216600 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216602 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216607 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216610 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216613 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216619 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.219309 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216622 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216625 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216628 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216630 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216634 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216637 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216639 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216642 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216644 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216647 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216649 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216651 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216654 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216657 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216659 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216662 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216664 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216667 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.216669 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217049 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.219758 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217054 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217057 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217060 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217064 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217068 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217071 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217074 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217077 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217080 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217083 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217085 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217089 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217091 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217094 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217097 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217100 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217103 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217105 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217108 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.220250 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217111 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217113 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217116 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217118 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217120 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217123 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217125 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217128 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217130 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217133 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217137 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217139 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217141 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217144 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217147 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217149 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217152 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217154 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217157 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217159 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217162 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.220727 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217165 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217167 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217170 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217172 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217175 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217178 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217180 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217183 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217186 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217188 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217191 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217193 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217197 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217201 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217204 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217206 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217209 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217212 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217215 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.221257 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217217 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217220 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217223 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217226 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217229 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217231 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217234 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217236 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217239 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217242 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217245 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217248 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217251 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217254 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217257 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217260 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217262 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217265 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217267 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.221719 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217270 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217273 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217275 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217278 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217280 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217283 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217286 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217288 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217290 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217293 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217295 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217298 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217300 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217302 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217305 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217307 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217310 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217313 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217315 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.217318 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217397 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 13:16:26.222220 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217404 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217410 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217414 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217419 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217422 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217426 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217430 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217433 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217436 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217440 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217443 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217447 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217450 2573 flags.go:64] FLAG: --cgroup-root="" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217453 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217456 2573 flags.go:64] FLAG: --client-ca-file="" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217459 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217462 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217467 2573 flags.go:64] FLAG: --cluster-domain="" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217470 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217473 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217476 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217479 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217483 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 13:16:26.222736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217486 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217489 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217492 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217495 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217497 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217500 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217503 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217506 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217516 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217519 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217521 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217524 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217527 2573 flags.go:64] FLAG: --enable-server="true" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217530 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217535 2573 flags.go:64] FLAG: --event-burst="100" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217538 2573 flags.go:64] FLAG: --event-qps="50" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217541 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217544 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217547 2573 flags.go:64] FLAG: --eviction-hard="" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217550 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217553 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217556 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217559 2573 flags.go:64] FLAG: --eviction-soft="" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217562 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217565 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 13:16:26.223331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217568 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217570 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217573 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217576 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217579 2573 flags.go:64] FLAG: --feature-gates="" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217582 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217585 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217588 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217591 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217594 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217597 2573 flags.go:64] FLAG: --help="false" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217600 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217603 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217606 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217608 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217613 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217617 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217620 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217623 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217626 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217629 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217631 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217635 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217637 2573 flags.go:64] FLAG: --kube-reserved="" Jul 07 13:16:26.223953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217640 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217643 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217646 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217649 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217651 2573 flags.go:64] FLAG: --lock-file="" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217655 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217658 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217661 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217666 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217669 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217671 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217674 2573 flags.go:64] FLAG: --logging-format="text" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217677 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217680 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217683 2573 flags.go:64] FLAG: --manifest-url="" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217685 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217690 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217692 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217697 2573 flags.go:64] FLAG: --max-pods="110" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217700 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217703 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217705 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217708 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217715 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217717 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 13:16:26.224546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217721 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217728 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217731 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217734 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217737 2573 flags.go:64] FLAG: --pod-cidr="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217740 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217745 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217748 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217751 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217754 2573 flags.go:64] FLAG: --port="10250" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217757 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217760 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d5c32c9442067003" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217763 2573 flags.go:64] FLAG: --qos-reserved="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217767 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217770 2573 flags.go:64] FLAG: --register-node="true" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217772 2573 flags.go:64] FLAG: --register-with-taints="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217781 2573 flags.go:64] FLAG: --registry-burst="10" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217783 2573 flags.go:64] FLAG: --registry-qps="5" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217786 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217789 2573 flags.go:64] FLAG: --reserved-memory="" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217793 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217796 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217798 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217801 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217804 2573 flags.go:64] FLAG: --runonce="false" Jul 07 13:16:26.225193 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217806 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217810 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217813 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217816 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217819 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217822 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217826 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217829 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217832 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217835 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217838 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217841 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217845 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217848 2573 flags.go:64] FLAG: --system-cgroups="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217851 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217856 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217859 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217862 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217867 2573 flags.go:64] FLAG: --tls-min-version="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217870 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217873 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217876 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217879 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217882 2573 flags.go:64] FLAG: --v="2" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217886 2573 flags.go:64] FLAG: --version="false" Jul 07 13:16:26.225800 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217889 2573 flags.go:64] FLAG: --vmodule="" Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217893 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.217896 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218039 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218043 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218046 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218049 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218052 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218055 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218058 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218060 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218063 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218066 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218069 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218073 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218076 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218079 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218083 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218086 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218088 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.226420 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218091 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218093 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218096 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218098 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218101 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218103 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218106 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218108 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218111 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218114 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218116 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218119 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218121 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218124 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218127 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218129 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218132 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218134 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218137 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218139 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.226950 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218141 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218144 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218146 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218149 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218151 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218154 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218157 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218161 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218164 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218169 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218172 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218175 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218178 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218180 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218183 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218185 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218188 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218191 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218193 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.227437 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218196 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218198 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218201 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218207 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218210 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218213 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218215 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218218 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218220 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218223 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218225 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218234 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218236 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218239 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218242 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218244 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218247 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218250 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218252 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218254 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.227885 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218257 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218260 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218263 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218266 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218268 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218271 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218273 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218276 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218278 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218282 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218285 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218288 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218291 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218293 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218296 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218299 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218302 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218305 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218308 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.228390 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218310 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218312 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218315 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.218317 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.219493 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.226312 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.226331 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.227016 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.227023 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227088 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227094 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227098 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227101 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227104 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227106 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.228854 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227109 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227112 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227115 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227119 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227123 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227126 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227129 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227132 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227135 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227138 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227141 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227144 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227147 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227150 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227152 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227155 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227158 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227161 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227164 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.229283 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227166 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227169 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227171 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227174 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227184 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227187 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227190 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227192 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227195 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227197 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227200 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227204 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227207 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227210 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227213 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227215 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227218 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227221 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227223 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227226 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.229734 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227229 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227231 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227234 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227237 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227239 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227241 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227244 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227246 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227249 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227251 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227254 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227256 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227258 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227261 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227264 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227266 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227269 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227279 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227282 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.230253 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227284 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227287 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227289 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227292 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227295 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227297 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227300 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227302 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227305 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227307 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227309 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227312 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227314 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227317 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227319 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227321 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227324 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227326 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227329 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227332 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.230752 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227334 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227337 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227339 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227342 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227344 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227346 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227349 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227352 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227355 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227357 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227360 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227368 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227371 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227373 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227376 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.231245 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.227381 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227527 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227532 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227535 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227538 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227541 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227544 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227547 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227550 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227553 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227556 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227558 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227561 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227564 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227567 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227569 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227572 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227574 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227577 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227580 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 13:16:26.231717 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227582 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227585 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227587 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227590 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227592 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227595 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227598 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227600 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227603 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227611 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227615 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227617 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227619 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227622 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227624 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227627 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227629 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227632 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227634 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227637 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 13:16:26.232244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227639 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227642 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227644 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227647 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227649 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227652 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227654 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227658 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227662 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227665 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227668 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227671 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227674 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227676 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227679 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227681 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227684 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227686 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227689 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227692 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 13:16:26.232805 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227694 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227697 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227705 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227708 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227710 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227713 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227715 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227718 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227720 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227722 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227725 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227727 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227730 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227732 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227735 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227737 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227739 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227742 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227744 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 13:16:26.233380 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227748 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227752 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227755 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227758 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227760 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227763 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227766 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227769 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227772 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227776 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227779 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227782 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227785 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227787 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227790 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227793 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227801 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227804 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227806 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 13:16:26.234045 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227809 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:26.227811 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.227816 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.227967 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.230106 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.231431 2573 server.go:1019] "Starting client certificate rotation" Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.231567 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:16:26.234780 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.231622 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 13:16:26.236205 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.236184 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:16:26.238641 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.238627 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 13:16:26.247326 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.247311 2573 log.go:25] "Validated CRI v1 runtime API" Jul 07 13:16:26.249323 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.249308 2573 log.go:25] "Validated CRI v1 image API" Jul 07 13:16:26.250865 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.250851 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 13:16:26.253218 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.253199 2573 fs.go:135] Filesystem UUIDs: map[40ebd382-4e99-49a1-b18e-fc7a7890abb9:/dev/nvme0n1p3 6f204971-f74c-470c-826a-6007a8b1f588:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 13:16:26.253274 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.253218 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 13:16:26.258878 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.258763 2573 manager.go:216] Machine: {Timestamp:2026-07-07 13:16:26.257540834 +0000 UTC m=+0.322438800 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099469 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec287bc06be9d0d8ddf82b788fbde18f SystemUUID:ec287bc0-6be9-d0d8-ddf8-2b788fbde18f BootID:25e3a113-415e-4cec-bf4e-fa19d7888938 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:09:7c:05:fc:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:09:7c:05:fc:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:47:51:a5:43:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 13:16:26.258878 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.258873 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 13:16:26.258989 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.258966 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 13:16:26.261653 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.261635 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:16:26.261726 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.261699 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 13:16:26.261940 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.261735 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 13:16:26.262435 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.262425 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 13:16:26.262471 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.262438 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 13:16:26.262471 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.262445 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:16:26.263006 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.262996 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 13:16:26.264543 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.264531 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 13:16:26.267572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267560 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:16:26.267689 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267680 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 13:16:26.267729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267718 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 07 13:16:26.267762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267730 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 13:16:26.267762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267742 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 13:16:26.267762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267754 2573 kubelet.go:398] "Adding apiserver pod source" Jul 07 13:16:26.267762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.267763 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 13:16:26.268306 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.268293 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:16:26.268352 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.268315 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 13:16:26.270415 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.270397 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 13:16:26.272390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.272376 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 13:16:26.272459 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.272407 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 13:16:26.274439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274419 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274445 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274455 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274464 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274486 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274495 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274507 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 13:16:26.274525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274514 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 13:16:26.274810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274532 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 13:16:26.274810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274540 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 13:16:26.274810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274553 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 13:16:26.274810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.274561 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 13:16:26.275325 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.275303 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 13:16:26.275325 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.275322 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 13:16:26.278273 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.278254 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 13:16:26.278806 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.278779 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 13:16:26.278889 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.278780 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 13:16:26.279278 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.279263 2573 server.go:1268] "Started kubelet" Jul 07 13:16:26.279407 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.279320 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 13:16:26.279588 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.279551 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 13:16:26.279643 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.279613 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 13:16:26.280303 ip-10-0-133-20 systemd[1]: Started Kubernetes Kubelet. Jul 07 13:16:26.281361 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.281109 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 13:16:26.281671 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.281654 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 07 13:16:26.283661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.283638 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lx2k5" Jul 07 13:16:26.283992 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.283973 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 13:16:26.287200 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287180 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 13:16:26.287200 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287189 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 13:16:26.287352 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.287273 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.287715 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287694 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 13:16:26.287793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287707 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 13:16:26.287793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287730 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 13:16:26.287906 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287893 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 07 13:16:26.287906 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.287906 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 07 13:16:26.288248 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288221 2573 factory.go:55] Registering systemd factory Jul 07 13:16:26.288248 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288237 2573 factory.go:223] Registration of the systemd container factory successfully Jul 07 13:16:26.288505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288490 2573 factory.go:153] Registering CRI-O factory Jul 07 13:16:26.288505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288506 2573 factory.go:223] Registration of the crio container factory successfully Jul 07 13:16:26.288602 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288547 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 13:16:26.288602 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288588 2573 factory.go:103] Registering Raw factory Jul 07 13:16:26.288656 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.288614 2573 manager.go:1196] Started watching for new ooms in manager Jul 07 13:16:26.289190 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.289174 2573 manager.go:349] Starting recovery of all containers Jul 07 13:16:26.289484 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.288501 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-20.ec2.internal.18c0044e2ee1ed10 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-20.ec2.internal,UID:ip-10-0-133-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-20.ec2.internal,},FirstTimestamp:2026-07-07 13:16:26.279234832 +0000 UTC m=+0.344132774,LastTimestamp:2026-07-07 13:16:26.279234832 +0000 UTC m=+0.344132774,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-20.ec2.internal,}" Jul 07 13:16:26.292901 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.292880 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lx2k5" Jul 07 13:16:26.299776 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.299759 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:26.300252 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.299938 2573 manager.go:354] Recovery completed Jul 07 13:16:26.303573 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.303554 2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-20.ec2.internal\" not found" node="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.305959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.305942 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.308150 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308134 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.308222 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308164 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.308222 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308174 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.308710 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308698 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.308777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308699 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 13:16:26.308777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308762 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 13:16:26.308883 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.308788 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 07 13:16:26.309394 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.309375 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.309476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.309410 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.309476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.309425 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.310025 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.310012 2573 policy_none.go:49] "None policy: Start" Jul 07 13:16:26.310092 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.310032 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 13:16:26.310092 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.310045 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 13:16:26.311004 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.310991 2573 policy_none.go:47] "Start" Jul 07 13:16:26.350136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350119 2573 manager.go:337] "Starting Device Plugin manager" Jul 07 13:16:26.350233 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.350167 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 13:16:26.350233 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350181 2573 server.go:87] "Starting device plugin registration server" Jul 07 13:16:26.350432 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350419 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 13:16:26.350489 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350434 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 13:16:26.350562 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350539 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 13:16:26.350664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350645 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 13:16:26.350738 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.350689 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 13:16:26.351188 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.351168 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 13:16:26.351290 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.351214 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.424887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.424855 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 13:16:26.426181 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.426162 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 13:16:26.426277 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.426199 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 13:16:26.426277 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.426222 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 13:16:26.426358 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.426312 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 13:16:26.428594 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.428576 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:26.450899 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.450876 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.451741 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.451724 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.451840 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.451760 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.451840 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.451775 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.451840 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.451806 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.459753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.459711 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.459753 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.459734 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-20.ec2.internal\": node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.473222 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.473201 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.529012 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.528983 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal"] Jul 07 13:16:26.529097 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.529046 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.530502 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.530487 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.530586 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.530514 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.530586 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.530525 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.531790 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.531777 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.531902 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.531887 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.531956 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.531947 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.538287 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538271 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.538388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538299 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.538388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538317 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.538388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538341 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.538388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538362 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.538388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.538372 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.539657 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.539640 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.539736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.539668 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 13:16:26.540367 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.540348 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientMemory" Jul 07 13:16:26.540434 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.540375 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 13:16:26.540434 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.540386 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeHasSufficientPID" Jul 07 13:16:26.552555 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.552535 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-20.ec2.internal\" not found" node="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.556805 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.556790 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-20.ec2.internal\" not found" node="ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.574240 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.574222 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.589787 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.589767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b69ec39edf2c7e27054c97d97fb57365-config\") pod \"kube-apiserver-proxy-ip-10-0-133-20.ec2.internal\" (UID: \"b69ec39edf2c7e27054c97d97fb57365\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.589844 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.589793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.589844 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.589812 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.674723 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.674681 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.690107 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690084 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.690165 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690104 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.690165 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690133 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.690165 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690156 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b69ec39edf2c7e27054c97d97fb57365-config\") pod \"kube-apiserver-proxy-ip-10-0-133-20.ec2.internal\" (UID: \"b69ec39edf2c7e27054c97d97fb57365\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.690261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690180 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b69ec39edf2c7e27054c97d97fb57365-config\") pod \"kube-apiserver-proxy-ip-10-0-133-20.ec2.internal\" (UID: \"b69ec39edf2c7e27054c97d97fb57365\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.690261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.690224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/374450171b78f3634cbc0b2792a20e88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal\" (UID: \"374450171b78f3634cbc0b2792a20e88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.774877 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.774798 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.860183 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.860147 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.860311 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:26.860147 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:26.875512 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.875374 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:26.979633 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:26.979591 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:27.087029 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:27.086935 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:27.194372 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:27.194319 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:27.233471 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.233442 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 13:16:27.234169 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.233605 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 13:16:27.234169 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.233625 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 13:16:27.294515 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:27.294477 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-20.ec2.internal\" not found" Jul 07 13:16:27.295676 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.295651 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 13:11:26 +0000 UTC" deadline="2027-12-24 19:09:54.752886439 +0000 UTC" Jul 07 13:16:27.295727 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.295676 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12845h53m27.457213315s" Jul 07 13:16:27.351970 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.351900 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:27.389760 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.389732 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" Jul 07 13:16:27.406838 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.406815 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:16:27.409002 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.408969 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" Jul 07 13:16:27.416734 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.416715 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 13:16:27.561458 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:27.561426 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69ec39edf2c7e27054c97d97fb57365.slice/crio-a9c72be5f60eb6dd359d8cecb36563aa81342aec1974be1d8ab3055bed7034db WatchSource:0}: Error finding container a9c72be5f60eb6dd359d8cecb36563aa81342aec1974be1d8ab3055bed7034db: Status 404 returned error can't find the container with id a9c72be5f60eb6dd359d8cecb36563aa81342aec1974be1d8ab3055bed7034db Jul 07 13:16:27.565578 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.565563 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:16:27.696528 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:27.696487 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374450171b78f3634cbc0b2792a20e88.slice/crio-f1a538213f1d0cd3c3000d2b13d0af3083f8365f5654b89d15cc066839de0f37 WatchSource:0}: Error finding container f1a538213f1d0cd3c3000d2b13d0af3083f8365f5654b89d15cc066839de0f37: Status 404 returned error can't find the container with id f1a538213f1d0cd3c3000d2b13d0af3083f8365f5654b89d15cc066839de0f37 Jul 07 13:16:27.733694 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:27.733664 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:28.271072 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.271039 2573 apiserver.go:52] "Watching apiserver" Jul 07 13:16:28.282767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.282738 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 13:16:28.283872 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.283848 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-zllhn","openshift-multus/multus-5949d","openshift-multus/multus-additional-cni-plugins-qtd6q","openshift-multus/network-metrics-daemon-28m8c","kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal","openshift-network-diagnostics/network-check-target-dhtsw","openshift-network-operator/iptables-alerter-7ffnv","openshift-ovn-kubernetes/ovnkube-node-r2cmc","kube-system/global-pull-secret-syncer-qplbh","kube-system/konnectivity-agent-d4zs8","openshift-cluster-node-tuning-operator/tuned-l4mmq","openshift-dns/node-resolver-fnngb"] Jul 07 13:16:28.286797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.286773 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.288208 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.288188 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 13:16:28.288978 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.288960 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5949d" Jul 07 13:16:28.289820 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.289798 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 13:16:28.289820 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.289819 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zm4fg\"" Jul 07 13:16:28.289952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.289803 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.289952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.289809 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.291396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.291484 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291471 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.291730 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291613 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.291730 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291627 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 13:16:28.291730 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291692 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5hbkg\"" Jul 07 13:16:28.291892 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.291841 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 13:16:28.293625 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.293604 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.293718 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.293684 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:28.293718 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.293609 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 13:16:28.293984 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.293965 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 13:16:28.294063 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.294023 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-slpd7\"" Jul 07 13:16:28.296108 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.296088 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.298466 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.298448 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bvqnd\"" Jul 07 13:16:28.298560 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.298451 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.298560 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.298497 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 13:16:28.298698 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.298680 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:28.298774 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.298736 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:28.298837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.298827 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.299649 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6977b697-1c8a-4994-a3e0-1430604adb23-host\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.299649 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299657 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbj2\" (UniqueName: \"kubernetes.io/projected/6977b697-1c8a-4994-a3e0-1430604adb23-kube-api-access-gnbj2\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-system-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-cnibin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-hostroot\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-conf-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299799 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6977b697-1c8a-4994-a3e0-1430604adb23-serviceca\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.299837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-os-release\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-k8s-cni-cncf-io\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-multus\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-kubelet\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-multus-certs\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drtkf\" (UniqueName: \"kubernetes.io/projected/5e2e72de-b475-4cd9-86da-8a38b8567446-kube-api-access-drtkf\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-cni-binary-copy\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.299972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-socket-dir-parent\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.300006 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-bin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.300027 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-daemon-config\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.300066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-etc-kubernetes\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.300162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.300126 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-netns\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.301078 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.301044 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.301188 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.301095 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 13:16:28.303718 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.303693 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.303824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.303742 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.304285 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.304268 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.304369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.304303 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 13:16:28.304429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.304303 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-76m5v\"" Jul 07 13:16:28.306275 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.306250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.306391 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.306329 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:28.306391 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.306255 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d7vmg\"" Jul 07 13:16:28.306655 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.306634 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 13:16:28.306756 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.306701 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.306756 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.306747 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 13:16:28.307690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.307625 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.307690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.307639 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 13:16:28.307690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.307639 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 13:16:28.308658 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.308635 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.311081 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.311066 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 13:16:28.311901 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.311672 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 13:16:28.311901 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.311745 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5hgp5\"" Jul 07 13:16:28.312064 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.312029 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.314881 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.314859 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.315027 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.315011 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.315107 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.315028 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.315107 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.315032 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-svtlw\"" Jul 07 13:16:28.317669 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.317643 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 13:16:28.317771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.317655 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 13:16:28.317771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.317691 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-srdcm\"" Jul 07 13:16:28.324592 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.324574 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l9prf" Jul 07 13:16:28.337086 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.337063 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l9prf" Jul 07 13:16:28.389543 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.389519 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 13:16:28.400478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-conf-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400475 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-multus-certs\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-cnibin\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.400640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-registration-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.400640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400555 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-conf-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400591 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-multus-certs\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400637 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-dbus\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-multus\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-script-lib\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400738 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-multus\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-kubernetes\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-host\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r45zj\" (UniqueName: \"kubernetes.io/projected/94429720-9b52-4f82-9be5-b4c160ae7731-kube-api-access-r45zj\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.400852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-bin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-etc-kubernetes\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400934 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-cni-bin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400945 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1ce783-9547-4f55-b50f-bc613272a3ae-iptables-alerter-script\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400966 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-etc-kubernetes\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.400996 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-var-lib-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401037 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovn-node-metrics-cert\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4q8c\" (UniqueName: \"kubernetes.io/projected/4c5abd97-3186-4834-a29d-5cc52ed6c205-kube-api-access-q4q8c\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6977b697-1c8a-4994-a3e0-1430604adb23-host\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401105 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-hostroot\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401124 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6977b697-1c8a-4994-a3e0-1430604adb23-host\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.401154 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q42ff\" (UniqueName: \"kubernetes.io/projected/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kube-api-access-q42ff\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-hostroot\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401173 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-modprobe-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401217 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-run\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401235 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401280 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-kubelet-config\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401298 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7nr2\" (UniqueName: \"kubernetes.io/projected/6f5f807f-62cd-4965-a59e-952b2e12356c-kube-api-access-p7nr2\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-systemd-units\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401335 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-netd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401358 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebc4978-2fc4-4c32-b612-960ffd5bd966-agent-certs\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401391 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lszzs\" (UniqueName: \"kubernetes.io/projected/c313195a-53b5-4a03-a686-d4740f5b112d-kube-api-access-lszzs\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-sys-fs\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401437 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-systemd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysconfig\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.401616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401491 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pczkj\" (UniqueName: \"kubernetes.io/projected/f038fc69-abff-4174-bfdf-01d488dd6efd-kube-api-access-pczkj\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401532 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-etc-tuned\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401546 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c313195a-53b5-4a03-a686-d4740f5b112d-tmp-dir\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-cnibin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-netns\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401618 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-cnibin\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401657 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-lib-modules\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-os-release\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-k8s-cni-cncf-io\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401745 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-os-release\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401769 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drtkf\" (UniqueName: \"kubernetes.io/projected/5e2e72de-b475-4cd9-86da-8a38b8567446-kube-api-access-drtkf\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401797 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-k8s-cni-cncf-io\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psc92\" (UniqueName: \"kubernetes.io/projected/2f1ce783-9547-4f55-b50f-bc613272a3ae-kube-api-access-psc92\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-etc-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-log-socket\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.401999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c313195a-53b5-4a03-a686-d4740f5b112d-hosts-file\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-socket-dir-parent\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.402369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1ce783-9547-4f55-b50f-bc613272a3ae-host-slash\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402052 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-env-overrides\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-var-lib-kubelet\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402101 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-socket-dir-parent\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402112 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-tmp\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402129 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-netns\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402186 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-run-netns\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebc4978-2fc4-4c32-b612-960ffd5bd966-konnectivity-ca\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402269 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbj2\" (UniqueName: \"kubernetes.io/projected/6977b697-1c8a-4994-a3e0-1430604adb23-kube-api-access-gnbj2\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-system-cni-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402397 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-conf\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6977b697-1c8a-4994-a3e0-1430604adb23-serviceca\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-kubelet\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403251 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-os-release\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402470 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-host-var-lib-kubelet\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402485 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-socket-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-kubelet\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-slash\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-config\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402596 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-cni-binary-copy\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-daemon-config\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-ovn\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-node-log\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-bin\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-sys\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402769 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-device-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402805 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6977b697-1c8a-4994-a3e0-1430604adb23-serviceca\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:28.403767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-systemd\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-system-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402917 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.402955 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-system-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.403022 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-cni-dir\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.403080 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-cni-binary-copy\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.404381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.403207 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e2e72de-b475-4cd9-86da-8a38b8567446-multus-daemon-config\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.411033 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.410861 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 13:16:28.414924 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.414884 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbj2\" (UniqueName: \"kubernetes.io/projected/6977b697-1c8a-4994-a3e0-1430604adb23-kube-api-access-gnbj2\") pod \"node-ca-zllhn\" (UID: \"6977b697-1c8a-4994-a3e0-1430604adb23\") " pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.415070 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.414895 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drtkf\" (UniqueName: \"kubernetes.io/projected/5e2e72de-b475-4cd9-86da-8a38b8567446-kube-api-access-drtkf\") pod \"multus-5949d\" (UID: \"5e2e72de-b475-4cd9-86da-8a38b8567446\") " pod="openshift-multus/multus-5949d" Jul 07 13:16:28.432114 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.432059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" event={"ID":"374450171b78f3634cbc0b2792a20e88","Type":"ContainerStarted","Data":"f1a538213f1d0cd3c3000d2b13d0af3083f8365f5654b89d15cc066839de0f37"} Jul 07 13:16:28.433481 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.433454 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" event={"ID":"b69ec39edf2c7e27054c97d97fb57365","Type":"ContainerStarted","Data":"a9c72be5f60eb6dd359d8cecb36563aa81342aec1974be1d8ab3055bed7034db"} Jul 07 13:16:28.487624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.487589 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:28.503761 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-modprobe-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-run\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503832 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-kubelet-config\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7nr2\" (UniqueName: \"kubernetes.io/projected/6f5f807f-62cd-4965-a59e-952b2e12356c-kube-api-access-p7nr2\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503882 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-systemd-units\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503885 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-run\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-netd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.503959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503949 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-modprobe-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503976 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-systemd-units\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.503992 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-kubelet-config\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504066 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-netd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebc4978-2fc4-4c32-b612-960ffd5bd966-agent-certs\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lszzs\" (UniqueName: \"kubernetes.io/projected/c313195a-53b5-4a03-a686-d4740f5b112d-kube-api-access-lszzs\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504143 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-sys-fs\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504169 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-systemd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504193 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysconfig\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pczkj\" (UniqueName: \"kubernetes.io/projected/f038fc69-abff-4174-bfdf-01d488dd6efd-kube-api-access-pczkj\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504237 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-sys-fs\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-etc-tuned\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504286 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c313195a-53b5-4a03-a686-d4740f5b112d-tmp-dir\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504318 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-netns\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504345 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-lib-modules\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psc92\" (UniqueName: \"kubernetes.io/projected/2f1ce783-9547-4f55-b50f-bc613272a3ae-kube-api-access-psc92\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.504422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-etc-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-log-socket\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c313195a-53b5-4a03-a686-d4740f5b112d-hosts-file\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysconfig\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504480 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1ce783-9547-4f55-b50f-bc613272a3ae-host-slash\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504506 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504526 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-systemd\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-env-overrides\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504619 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c313195a-53b5-4a03-a686-d4740f5b112d-tmp-dir\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504732 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-var-lib-kubelet\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504754 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1ce783-9547-4f55-b50f-bc613272a3ae-host-slash\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504765 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-tmp\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504819 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c313195a-53b5-4a03-a686-d4740f5b112d-hosts-file\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504846 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-netns\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.505253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504954 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-etc-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.504956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebc4978-2fc4-4c32-b612-960ffd5bd966-konnectivity-ca\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505017 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-log-socket\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505031 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-env-overrides\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505050 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-lib-modules\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505053 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505063 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-var-lib-kubelet\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-system-cni-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505209 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-conf\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-system-cni-dir\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-os-release\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505350 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-d\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-socket-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.506147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505398 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-sysctl-conf\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-kubelet\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505439 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-os-release\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-kubelet\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505498 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-slash\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505537 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-config\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505570 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-socket-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505587 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-slash\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebc4978-2fc4-4c32-b612-960ffd5bd966-konnectivity-ca\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.505716 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505755 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-ovn\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.505791 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:16:29.005768986 +0000 UTC m=+3.070666924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-node-log\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505812 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-run-ovn\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505840 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-bin\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-sys\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505907 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-device-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505932 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.505987 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-systemd\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506065 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-cnibin\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-registration-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-dbus\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506152 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-script-lib\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-kubernetes\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506149 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-config\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506122 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-systemd\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-host\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506188 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-cni-bin\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-registration-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506285 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7ec2ac7c-f972-4867-a382-81944fe7f93d-dbus\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f038fc69-abff-4174-bfdf-01d488dd6efd-cnibin\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r45zj\" (UniqueName: \"kubernetes.io/projected/94429720-9b52-4f82-9be5-b4c160ae7731-kube-api-access-r45zj\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.507853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506320 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-etc-kubernetes\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-sys\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506381 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-node-log\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506387 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94429720-9b52-4f82-9be5-b4c160ae7731-host\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506353 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-device-dir\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506383 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1ce783-9547-4f55-b50f-bc613272a3ae-iptables-alerter-script\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-var-lib-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.506549 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovn-node-metrics-cert\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.506619 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:16:29.006600441 +0000 UTC m=+3.071498379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-var-lib-openvswitch\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4q8c\" (UniqueName: \"kubernetes.io/projected/4c5abd97-3186-4834-a29d-5cc52ed6c205-kube-api-access-q4q8c\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506702 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506736 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovnkube-script-lib\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.508793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q42ff\" (UniqueName: \"kubernetes.io/projected/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kube-api-access-q42ff\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.509679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.506942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.509679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.507017 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c5abd97-3186-4834-a29d-5cc52ed6c205-host-run-ovn-kubernetes\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.509679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.507047 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1ce783-9547-4f55-b50f-bc613272a3ae-iptables-alerter-script\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.509679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.507200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f038fc69-abff-4174-bfdf-01d488dd6efd-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.509679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.508364 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-tmp\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.511027 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.510985 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94429720-9b52-4f82-9be5-b4c160ae7731-etc-tuned\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.513064 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.513039 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:28.513159 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.513068 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:28.513159 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.513083 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:28.513159 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:28.513156 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:16:29.013138624 +0000 UTC m=+3.078036575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:28.513335 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.513210 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c5abd97-3186-4834-a29d-5cc52ed6c205-ovn-node-metrics-cert\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.514407 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.514251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psc92\" (UniqueName: \"kubernetes.io/projected/2f1ce783-9547-4f55-b50f-bc613272a3ae-kube-api-access-psc92\") pod \"iptables-alerter-7ffnv\" (UID: \"2f1ce783-9547-4f55-b50f-bc613272a3ae\") " pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.514407 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.514306 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pczkj\" (UniqueName: \"kubernetes.io/projected/f038fc69-abff-4174-bfdf-01d488dd6efd-kube-api-access-pczkj\") pod \"multus-additional-cni-plugins-qtd6q\" (UID: \"f038fc69-abff-4174-bfdf-01d488dd6efd\") " pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.514940 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.514917 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lszzs\" (UniqueName: \"kubernetes.io/projected/c313195a-53b5-4a03-a686-d4740f5b112d-kube-api-access-lszzs\") pod \"node-resolver-fnngb\" (UID: \"c313195a-53b5-4a03-a686-d4740f5b112d\") " pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.515451 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.515399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7nr2\" (UniqueName: \"kubernetes.io/projected/6f5f807f-62cd-4965-a59e-952b2e12356c-kube-api-access-p7nr2\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:28.516338 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.516291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4q8c\" (UniqueName: \"kubernetes.io/projected/4c5abd97-3186-4834-a29d-5cc52ed6c205-kube-api-access-q4q8c\") pod \"ovnkube-node-r2cmc\" (UID: \"4c5abd97-3186-4834-a29d-5cc52ed6c205\") " pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.516882 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.516841 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebc4978-2fc4-4c32-b612-960ffd5bd966-agent-certs\") pod \"konnectivity-agent-d4zs8\" (UID: \"0ebc4978-2fc4-4c32-b612-960ffd5bd966\") " pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.517507 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.517486 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q42ff\" (UniqueName: \"kubernetes.io/projected/0f80c0c1-ad54-4cec-9ff9-aab50cb55f55-kube-api-access-q42ff\") pod \"aws-ebs-csi-driver-node-8xhtb\" (UID: \"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.517811 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.517787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r45zj\" (UniqueName: \"kubernetes.io/projected/94429720-9b52-4f82-9be5-b4c160ae7731-kube-api-access-r45zj\") pod \"tuned-l4mmq\" (UID: \"94429720-9b52-4f82-9be5-b4c160ae7731\") " pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.598005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.597905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zllhn" Jul 07 13:16:28.605072 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.605037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5949d" Jul 07 13:16:28.614393 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.614368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" Jul 07 13:16:28.615847 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.615803 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6977b697_1c8a_4994_a3e0_1430604adb23.slice/crio-9e44a3ad724b0b7b318e649336a20ec7431752593391e1e3e8445cda82a705ca WatchSource:0}: Error finding container 9e44a3ad724b0b7b318e649336a20ec7431752593391e1e3e8445cda82a705ca: Status 404 returned error can't find the container with id 9e44a3ad724b0b7b318e649336a20ec7431752593391e1e3e8445cda82a705ca Jul 07 13:16:28.619019 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.618925 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2e72de_b475_4cd9_86da_8a38b8567446.slice/crio-e306d4dc2612f4549487439e5a8b7dc0871830b711f7f78f47c2b43e18e0580a WatchSource:0}: Error finding container e306d4dc2612f4549487439e5a8b7dc0871830b711f7f78f47c2b43e18e0580a: Status 404 returned error can't find the container with id e306d4dc2612f4549487439e5a8b7dc0871830b711f7f78f47c2b43e18e0580a Jul 07 13:16:28.619958 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.619935 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" Jul 07 13:16:28.625202 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.625174 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf038fc69_abff_4174_bfdf_01d488dd6efd.slice/crio-d8f7860f08f2ca8683c85cbc6192fa5ef816b738103b9aaf31ba89999ef00b1f WatchSource:0}: Error finding container d8f7860f08f2ca8683c85cbc6192fa5ef816b738103b9aaf31ba89999ef00b1f: Status 404 returned error can't find the container with id d8f7860f08f2ca8683c85cbc6192fa5ef816b738103b9aaf31ba89999ef00b1f Jul 07 13:16:28.626354 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.626192 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ffnv" Jul 07 13:16:28.631244 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.631218 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f80c0c1_ad54_4cec_9ff9_aab50cb55f55.slice/crio-8d48f1226cc061be109635a9e1646a14ee5401ab0b20288339f1003cbd0b7c20 WatchSource:0}: Error finding container 8d48f1226cc061be109635a9e1646a14ee5401ab0b20288339f1003cbd0b7c20: Status 404 returned error can't find the container with id 8d48f1226cc061be109635a9e1646a14ee5401ab0b20288339f1003cbd0b7c20 Jul 07 13:16:28.632165 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.632117 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:28.637104 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.637072 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1ce783_9547_4f55_b50f_bc613272a3ae.slice/crio-45a9c2b2f63fc9db29ea8aa1ddaf05f21b641482ba6a749b6dfa8936fe1b0d45 WatchSource:0}: Error finding container 45a9c2b2f63fc9db29ea8aa1ddaf05f21b641482ba6a749b6dfa8936fe1b0d45: Status 404 returned error can't find the container with id 45a9c2b2f63fc9db29ea8aa1ddaf05f21b641482ba6a749b6dfa8936fe1b0d45 Jul 07 13:16:28.639203 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.639135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:28.643439 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.643413 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5abd97_3186_4834_a29d_5cc52ed6c205.slice/crio-80b5e3009c46732c41fcc46079d868916e977f18917e31076a5877eea18528ce WatchSource:0}: Error finding container 80b5e3009c46732c41fcc46079d868916e977f18917e31076a5877eea18528ce: Status 404 returned error can't find the container with id 80b5e3009c46732c41fcc46079d868916e977f18917e31076a5877eea18528ce Jul 07 13:16:28.645028 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.644842 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" Jul 07 13:16:28.649005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.648820 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fnngb" Jul 07 13:16:28.649448 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.649424 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebc4978_2fc4_4c32_b612_960ffd5bd966.slice/crio-da534b61e2f8a837cdb45cd401f442a51cd9210c472e99746d789ed0cc1c8aec WatchSource:0}: Error finding container da534b61e2f8a837cdb45cd401f442a51cd9210c472e99746d789ed0cc1c8aec: Status 404 returned error can't find the container with id da534b61e2f8a837cdb45cd401f442a51cd9210c472e99746d789ed0cc1c8aec Jul 07 13:16:28.659612 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.659587 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94429720_9b52_4f82_9be5_b4c160ae7731.slice/crio-94468e4c6291a812b5ea3ad5b304fc3fdc94fbe5448aef51d556ec0b31c537d0 WatchSource:0}: Error finding container 94468e4c6291a812b5ea3ad5b304fc3fdc94fbe5448aef51d556ec0b31c537d0: Status 404 returned error can't find the container with id 94468e4c6291a812b5ea3ad5b304fc3fdc94fbe5448aef51d556ec0b31c537d0 Jul 07 13:16:28.664513 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:16:28.664493 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc313195a_53b5_4a03_a686_d4740f5b112d.slice/crio-d425959af519eab696e33239877903cecb9bb8b07da8b8471e7b38bf6e0fb249 WatchSource:0}: Error finding container d425959af519eab696e33239877903cecb9bb8b07da8b8471e7b38bf6e0fb249: Status 404 returned error can't find the container with id d425959af519eab696e33239877903cecb9bb8b07da8b8471e7b38bf6e0fb249 Jul 07 13:16:28.669397 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:28.669377 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 13:16:29.011464 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.011370 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:29.011632 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.011540 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:29.011632 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.011605 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:16:30.011588852 +0000 UTC m=+4.076486786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:29.011632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.011545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:29.011835 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.011661 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:29.011835 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.011755 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:16:30.011737926 +0000 UTC m=+4.076635884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:29.112907 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.112858 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:29.113089 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.113063 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:29.113168 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.113091 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:29.113168 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.113102 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:29.113281 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:29.113174 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:16:30.113155856 +0000 UTC m=+4.178053793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:29.341088 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.340986 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:11:28 +0000 UTC" deadline="2028-02-18 11:24:45.140944623 +0000 UTC" Jul 07 13:16:29.341088 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.341025 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14182h8m15.799924979s" Jul 07 13:16:29.437667 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.437524 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" event={"ID":"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55","Type":"ContainerStarted","Data":"8d48f1226cc061be109635a9e1646a14ee5401ab0b20288339f1003cbd0b7c20"} Jul 07 13:16:29.439819 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.439732 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerStarted","Data":"d8f7860f08f2ca8683c85cbc6192fa5ef816b738103b9aaf31ba89999ef00b1f"} Jul 07 13:16:29.442870 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.442843 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"80b5e3009c46732c41fcc46079d868916e977f18917e31076a5877eea18528ce"} Jul 07 13:16:29.444804 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.444772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ffnv" event={"ID":"2f1ce783-9547-4f55-b50f-bc613272a3ae","Type":"ContainerStarted","Data":"45a9c2b2f63fc9db29ea8aa1ddaf05f21b641482ba6a749b6dfa8936fe1b0d45"} Jul 07 13:16:29.446654 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.446597 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5949d" event={"ID":"5e2e72de-b475-4cd9-86da-8a38b8567446","Type":"ContainerStarted","Data":"e306d4dc2612f4549487439e5a8b7dc0871830b711f7f78f47c2b43e18e0580a"} Jul 07 13:16:29.448752 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.448694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zllhn" event={"ID":"6977b697-1c8a-4994-a3e0-1430604adb23","Type":"ContainerStarted","Data":"9e44a3ad724b0b7b318e649336a20ec7431752593391e1e3e8445cda82a705ca"} Jul 07 13:16:29.450744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.450712 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fnngb" event={"ID":"c313195a-53b5-4a03-a686-d4740f5b112d","Type":"ContainerStarted","Data":"d425959af519eab696e33239877903cecb9bb8b07da8b8471e7b38bf6e0fb249"} Jul 07 13:16:29.452432 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.452382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" event={"ID":"94429720-9b52-4f82-9be5-b4c160ae7731","Type":"ContainerStarted","Data":"94468e4c6291a812b5ea3ad5b304fc3fdc94fbe5448aef51d556ec0b31c537d0"} Jul 07 13:16:29.453843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:29.453815 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d4zs8" event={"ID":"0ebc4978-2fc4-4c32-b612-960ffd5bd966","Type":"ContainerStarted","Data":"da534b61e2f8a837cdb45cd401f442a51cd9210c472e99746d789ed0cc1c8aec"} Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.021650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.021724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.021868 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.021933 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:16:32.021913398 +0000 UTC m=+6.086811335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.022403 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:30.022559 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.022506 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:16:32.022486488 +0000 UTC m=+6.087384435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:30.122815 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.122123 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:30.122815 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.122350 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:30.122815 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.122372 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:30.122815 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.122386 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:30.122815 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.122451 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:16:32.122431661 +0000 UTC m=+6.187329595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:30.342384 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.342174 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 13:11:28 +0000 UTC" deadline="2028-04-09 18:12:27.258165232 +0000 UTC" Jul 07 13:16:30.342384 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.342222 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15412h55m56.915949068s" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.435861 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.436038 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.436139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.436244 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:30.436312 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:30.436775 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:30.436385 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.045768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.045844 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.045990 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.046058 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:16:36.04603642 +0000 UTC m=+10.110934391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.046490 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:32.046631 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.046544 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:16:36.046527844 +0000 UTC m=+10.111425793 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:32.146636 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.146552 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:32.146820 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.146751 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:32.146820 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.146775 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:32.146820 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.146788 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:32.147020 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.146852 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:16:36.146831063 +0000 UTC m=+10.211728999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:32.427484 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.427388 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:32.427666 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.427552 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:32.427740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.427658 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:32.427988 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.427832 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:32.427988 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:32.427887 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:32.428198 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:32.427988 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:34.427152 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:34.427074 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:34.427569 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:34.427081 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:34.427569 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:34.427204 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:34.427569 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:34.427074 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:34.427569 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:34.427266 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:34.427569 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:34.427383 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:36.079400 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.079355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:36.079830 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.079420 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:36.079830 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.079532 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:36.079830 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.079532 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:36.079830 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.079602 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:16:44.079582529 +0000 UTC m=+18.144480468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:36.079830 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.079626 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:16:44.079613895 +0000 UTC m=+18.144511840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:36.180769 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.180725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:36.180983 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.180891 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:36.180983 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.180931 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:36.180983 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.180945 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:36.181163 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.181011 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:16:44.180990196 +0000 UTC m=+18.245888131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:36.428449 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.428350 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:36.428449 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.428356 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:36.428690 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.428474 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:36.428690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:36.428484 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:36.428690 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.428594 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:36.428838 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:36.428699 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:38.426449 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:38.426418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:38.426873 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:38.426418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:38.426873 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:38.426538 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:38.426873 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:38.426629 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:38.426873 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:38.426418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:38.426873 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:38.426790 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:40.426694 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.426559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:40.426694 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.426567 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:40.426694 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:40.426692 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:40.427716 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:40.426785 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:40.427716 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.426859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:40.427716 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:40.426943 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:40.479942 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.479484 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" event={"ID":"b69ec39edf2c7e27054c97d97fb57365","Type":"ContainerStarted","Data":"bd0b2828b31c8f1662b4e6690a4da2b530d2c40541ea849219367b526c31f559"} Jul 07 13:16:40.481548 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.481428 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" event={"ID":"94429720-9b52-4f82-9be5-b4c160ae7731","Type":"ContainerStarted","Data":"b30f9616a3ac0d7f43986a8b8f2032336d4de3d06df548cc689404fe44a212ed"} Jul 07 13:16:40.494533 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.494488 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-20.ec2.internal" podStartSLOduration=13.494474953 podStartE2EDuration="13.494474953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:40.493859915 +0000 UTC m=+14.558757891" watchObservedRunningTime="2026-07-07 13:16:40.494474953 +0000 UTC m=+14.559372903" Jul 07 13:16:40.510040 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:40.509969 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l4mmq" podStartSLOduration=2.414009179 podStartE2EDuration="13.509954456s" totalImagesPullingTime="11.095945277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:40.509720242 +0000 UTC m=+14.574618195" watchObservedRunningTime="2026-07-07 13:16:40.509954456 +0000 UTC m=+14.574852406" Jul 07 13:16:41.485061 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.484795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d4zs8" event={"ID":"0ebc4978-2fc4-4c32-b612-960ffd5bd966","Type":"ContainerStarted","Data":"ffc4db5c448b1dc77076e29926e3f6d8a62a6b1e46c56ea8a8cbc7e7a210961e"} Jul 07 13:16:41.486553 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.486521 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" event={"ID":"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55","Type":"ContainerStarted","Data":"af5a213374ae04dfd2380f11edb72228b1a9e0249232511d3e11b399ab735b30"} Jul 07 13:16:41.488074 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.488039 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerStarted","Data":"158e23f438f25eab74484bf40c40b41a476b4df5be3b67d0a05e7e077ad28050"} Jul 07 13:16:41.489611 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.489584 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ffnv" event={"ID":"2f1ce783-9547-4f55-b50f-bc613272a3ae","Type":"ContainerStarted","Data":"6aadb3d466ee3b80553be5910867cd1efff7863c80de4c6450bd3ed99ad3231e"} Jul 07 13:16:41.491188 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.491165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zllhn" event={"ID":"6977b697-1c8a-4994-a3e0-1430604adb23","Type":"ContainerStarted","Data":"12f358b86b080fcf036360665abb5f2cc260aaa2e6730bb22391f3dd065f9beb"} Jul 07 13:16:41.492621 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.492592 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" event={"ID":"374450171b78f3634cbc0b2792a20e88","Type":"ContainerStarted","Data":"e0a10ad64ebf68097f92358b20d6a6921c6f3ad3c2e5ef70c9f965eeaf127ebf"} Jul 07 13:16:41.494256 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.494235 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fnngb" event={"ID":"c313195a-53b5-4a03-a686-d4740f5b112d","Type":"ContainerStarted","Data":"893bbd5de7c49bb474e95d307073e7c4c27a134b779173010fa873a2232e30e5"} Jul 07 13:16:41.500227 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.500184 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d4zs8" podStartSLOduration=3.4133728740000002 podStartE2EDuration="14.500168334s" totalImagesPullingTime="11.08679546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:41.499388803 +0000 UTC m=+15.564286787" watchObservedRunningTime="2026-07-07 13:16:41.500168334 +0000 UTC m=+15.565066290" Jul 07 13:16:41.548264 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.548206 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7ffnv" podStartSLOduration=3.414879978 podStartE2EDuration="14.548187022s" totalImagesPullingTime="11.133307044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:41.547707079 +0000 UTC m=+15.612605052" watchObservedRunningTime="2026-07-07 13:16:41.548187022 +0000 UTC m=+15.613084974" Jul 07 13:16:41.562381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.562331 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fnngb" podStartSLOduration=3.487590414 podStartE2EDuration="14.562313041s" totalImagesPullingTime="11.074722627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:41.5614041 +0000 UTC m=+15.626302051" watchObservedRunningTime="2026-07-07 13:16:41.562313041 +0000 UTC m=+15.627210997" Jul 07 13:16:41.574936 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.574869 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zllhn" podStartSLOduration=3.451639282 podStartE2EDuration="14.574852072s" totalImagesPullingTime="11.12321279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:41.574583493 +0000 UTC m=+15.639481460" watchObservedRunningTime="2026-07-07 13:16:41.574852072 +0000 UTC m=+15.639750024" Jul 07 13:16:41.665037 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.665002 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:41.665691 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:41.665665 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:42.426924 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:42.426876 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:42.426924 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:42.426898 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:42.427178 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:42.427020 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:42.427178 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:42.427033 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:42.427178 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:42.427125 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:42.427326 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:42.427224 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:42.496991 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:42.496962 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:42.497425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:42.497043 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d4zs8" Jul 07 13:16:44.145782 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.145742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:44.146329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.145825 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:44.146329 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.145933 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:44.146329 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.145971 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:44.146329 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.146017 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:00.1459952 +0000 UTC m=+34.210893145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:16:44.146329 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.146039 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:17:00.146028716 +0000 UTC m=+34.210926648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:16:44.246804 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.246770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:44.247009 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.246981 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:16:44.247009 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.247008 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:16:44.247124 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.247021 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:44.247124 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.247085 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:00.247065364 +0000 UTC m=+34.311963307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.427176 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.427198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:44.427190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.427312 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.427439 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:44.427618 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:44.427507 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:45.503532 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:45.503502 2573 generic.go:356] "Generic (PLEG): container finished" podID="374450171b78f3634cbc0b2792a20e88" containerID="e0a10ad64ebf68097f92358b20d6a6921c6f3ad3c2e5ef70c9f965eeaf127ebf" exitCode=0 Jul 07 13:16:45.503895 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:45.503546 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" event={"ID":"374450171b78f3634cbc0b2792a20e88","Type":"ContainerDied","Data":"e0a10ad64ebf68097f92358b20d6a6921c6f3ad3c2e5ef70c9f965eeaf127ebf"} Jul 07 13:16:46.429847 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:46.429819 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:46.430069 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:46.429934 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:46.430069 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:46.429944 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:46.430069 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:46.429964 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:46.430069 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:46.430022 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:46.430272 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:46.430111 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:48.430283 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:48.430257 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:48.430789 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:48.430364 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:48.430789 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:48.430360 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:48.430789 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:48.430370 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:48.430789 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:48.430489 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:48.430789 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:48.430589 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:49.491028 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.490844 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 13:16:49.517007 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.516946 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" event={"ID":"374450171b78f3634cbc0b2792a20e88","Type":"ContainerStarted","Data":"f99d0b2f44e9a70ceb747eb3d69a04ef1312fafed875ef036deebc8f604c4420"} Jul 07 13:16:49.519254 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.519215 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" event={"ID":"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55","Type":"ContainerStarted","Data":"6da27864fe5441f2b140bd657a49aa17ec11ddc444c89c62be5c8962440af1e8"} Jul 07 13:16:49.521454 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.521424 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="158e23f438f25eab74484bf40c40b41a476b4df5be3b67d0a05e7e077ad28050" exitCode=0 Jul 07 13:16:49.521534 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.521481 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"158e23f438f25eab74484bf40c40b41a476b4df5be3b67d0a05e7e077ad28050"} Jul 07 13:16:49.523590 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.523547 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"148c1b605058ab9af9093dcc14de4c734ce76f65e1e7bdeb5801493b17f7971a"} Jul 07 13:16:49.523688 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.523631 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"169cea0370b8a23c01acd7bc9168ca13d3fb7da38511737f57b71c834f39d8b6"} Jul 07 13:16:49.525388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.525367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5949d" event={"ID":"5e2e72de-b475-4cd9-86da-8a38b8567446","Type":"ContainerStarted","Data":"5ad15375c9e51d765feebf9aff8989e3ba6be9abe28e93fbaf11b3340fd3d824"} Jul 07 13:16:49.534651 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.534601 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-20.ec2.internal" podStartSLOduration=22.534586914 podStartE2EDuration="22.534586914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:49.534233561 +0000 UTC m=+23.599131513" watchObservedRunningTime="2026-07-07 13:16:49.534586914 +0000 UTC m=+23.599484865" Jul 07 13:16:49.572315 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:49.572266 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5949d" podStartSLOduration=1.848642156 podStartE2EDuration="22.572247266s" totalImagesPullingTime="20.72360511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:49.572107176 +0000 UTC m=+23.637005127" watchObservedRunningTime="2026-07-07 13:16:49.572247266 +0000 UTC m=+23.637145218" Jul 07 13:16:50.365695 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.365596 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T13:16:49.491025728Z","UUID":"8ed00b88-7035-46ee-a249-19a24240acf5","Handler":null,"Name":"","Endpoint":""} Jul 07 13:16:50.367457 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.367433 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 13:16:50.367595 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.367477 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 13:16:50.430842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.430813 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:50.431037 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:50.430954 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:50.431037 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.430992 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:50.431167 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:50.431062 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:50.431167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.431111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:50.431276 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:50.431185 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:50.532629 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.532591 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"a1d57303419dfc9344eb8f5201ba544d3348d1e5ac988f854c852c42a755b4a4"} Jul 07 13:16:50.533309 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.532682 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"c7a31dfd277b429666aafd4ce9a2173f051b3d372d3feb70f69ec4a5baaddfcd"} Jul 07 13:16:50.533309 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.532714 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"adc5d955968c5aebca1590d6c049b802092d676f65672bdbb1cc476401bcc75a"} Jul 07 13:16:50.533309 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:50.532746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"d3a83f56c798f21ae1d857a254b5b068c617e23b17fe1c255126c141a1f201e2"} Jul 07 13:16:51.537014 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:51.536979 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" event={"ID":"0f80c0c1-ad54-4cec-9ff9-aab50cb55f55","Type":"ContainerStarted","Data":"3c446f24802b32a071a389e75610c6275572eca1fdc0f12538f3fe1f95d6f200"} Jul 07 13:16:51.555892 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:51.555840 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xhtb" podStartSLOduration=4.22207912 podStartE2EDuration="24.555820375s" totalImagesPullingTime="20.333741255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:51.555485485 +0000 UTC m=+25.620383436" watchObservedRunningTime="2026-07-07 13:16:51.555820375 +0000 UTC m=+25.620718328" Jul 07 13:16:52.427369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:52.427334 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:52.427557 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:52.427377 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:52.427557 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:52.427343 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:52.427557 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:52.427453 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:52.427712 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:52.427550 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:52.427712 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:52.427597 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:52.543505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:52.543468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"2420ead5d5076717a78e08e0bc6292be978af16e79e8941ebd096800db7e7f25"} Jul 07 13:16:54.426973 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:54.426743 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:54.427438 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:54.426754 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:54.427438 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:54.427086 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:54.427438 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:54.426785 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:54.427438 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:54.427185 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:54.427438 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:54.427277 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:55.551676 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.551592 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="807af9694983139207722f8cf2ff87459c9d589aef7e7a51b4061ec945bda76b" exitCode=0 Jul 07 13:16:55.551676 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.551660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"807af9694983139207722f8cf2ff87459c9d589aef7e7a51b4061ec945bda76b"} Jul 07 13:16:55.555396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.555372 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" event={"ID":"4c5abd97-3186-4834-a29d-5cc52ed6c205","Type":"ContainerStarted","Data":"9382040a83fc902d88a402bee688927240b4a5c598da24f02180d9c1f7784fc2"} Jul 07 13:16:55.555664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.555644 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:55.555769 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.555704 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:55.555769 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.555746 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:55.571422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.571400 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:55.571528 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.571490 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:16:55.597366 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:55.597313 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" podStartSLOduration=7.898456373 podStartE2EDuration="28.597298708s" totalImagesPullingTime="20.698842335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:16:55.595876752 +0000 UTC m=+29.660774703" watchObservedRunningTime="2026-07-07 13:16:55.597298708 +0000 UTC m=+29.662196659" Jul 07 13:16:56.428136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:56.428101 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:56.428279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:56.428190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:56.428279 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:56.428211 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:56.428279 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:56.428271 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:56.428422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:56.428340 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:56.428498 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:56.428475 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:57.227739 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.227570 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qplbh"] Jul 07 13:16:57.228405 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.227817 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:57.228405 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:57.227927 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:57.231309 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.231285 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhtsw"] Jul 07 13:16:57.231451 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.231365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:57.231451 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:57.231440 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:57.232076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.232054 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-28m8c"] Jul 07 13:16:57.232176 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.232158 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:57.232250 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:57.232234 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:57.561219 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.561186 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="bafc0470aa408de9ccdc358bf8c396362eb387c41bcdca7756152ccd17d153e9" exitCode=0 Jul 07 13:16:57.563460 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:57.561510 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"bafc0470aa408de9ccdc358bf8c396362eb387c41bcdca7756152ccd17d153e9"} Jul 07 13:16:58.426846 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:58.426811 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:16:58.427245 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:58.426935 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:16:58.427245 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:58.426939 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:16:58.427245 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:58.427008 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:16:59.427078 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:59.427052 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:16:59.427511 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:16:59.427161 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:16:59.567132 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:59.567100 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="08917002af6b8cb9245cb0d3bbc30743768a81510c9ebd428002fce6ecf13b02" exitCode=0 Jul 07 13:16:59.567282 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:16:59.567153 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"08917002af6b8cb9245cb0d3bbc30743768a81510c9ebd428002fce6ecf13b02"} Jul 07 13:17:00.164755 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:00.164723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:00.164961 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:00.164782 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:00.164961 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.164865 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 13:17:00.164961 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.164904 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:17:00.164961 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.164944 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret podName:7ec2ac7c-f972-4867-a382-81944fe7f93d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:32.164928881 +0000 UTC m=+66.229826827 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret") pod "global-pull-secret-syncer-qplbh" (UID: "7ec2ac7c-f972-4867-a382-81944fe7f93d") : object "kube-system"/"original-pull-secret" not registered Jul 07 13:17:00.165160 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.164968 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs podName:6f5f807f-62cd-4965-a59e-952b2e12356c nodeName:}" failed. No retries permitted until 2026-07-07 13:17:32.164952496 +0000 UTC m=+66.229850439 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs") pod "network-metrics-daemon-28m8c" (UID: "6f5f807f-62cd-4965-a59e-952b2e12356c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 13:17:00.266177 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.266141 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 13:17:00.266177 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.266177 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 13:17:00.266474 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.266192 2573 projected.go:196] Error preparing data for projected volume kube-api-access-glct6 for pod openshift-network-diagnostics/network-check-target-dhtsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:17:00.266474 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.266256 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6 podName:d0cd60f0-ab70-4ee8-8e1f-15445dc32906 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:32.266238083 +0000 UTC m=+66.331136031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-glct6" (UniqueName: "kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6") pod "network-check-target-dhtsw" (UID: "d0cd60f0-ab70-4ee8-8e1f-15445dc32906") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 13:17:00.266936 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:00.266892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:00.427285 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:00.427204 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:00.427650 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:00.427204 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:00.427650 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.427323 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:17:00.427650 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:00.427412 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:17:01.426950 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:01.426705 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:01.427136 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:01.427092 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-28m8c" podUID="6f5f807f-62cd-4965-a59e-952b2e12356c" Jul 07 13:17:02.427249 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:02.427214 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:02.427672 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:02.427214 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:02.427672 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:02.427345 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qplbh" podUID="7ec2ac7c-f972-4867-a382-81944fe7f93d" Jul 07 13:17:02.427672 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:02.427452 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dhtsw" podUID="d0cd60f0-ab70-4ee8-8e1f-15445dc32906" Jul 07 13:17:03.199275 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.199239 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-20.ec2.internal" event="NodeReady" Jul 07 13:17:03.199449 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.199410 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 13:17:03.231432 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.231401 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85cf8895b5-gt29s"] Jul 07 13:17:03.235406 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.235386 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.238170 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.238151 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 13:17:03.238331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.238176 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 13:17:03.238331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.238192 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x9hsn\"" Jul 07 13:17:03.238834 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.238532 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 13:17:03.242760 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.242713 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r6jmp"] Jul 07 13:17:03.246929 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.246712 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 13:17:03.248328 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.248310 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cwsnb"] Jul 07 13:17:03.248434 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.248410 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.251103 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.250897 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 13:17:03.251103 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.250900 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 13:17:03.251103 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.251003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9d22t\"" Jul 07 13:17:03.251392 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.251375 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 13:17:03.251930 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.251890 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85cf8895b5-gt29s"] Jul 07 13:17:03.252057 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.252018 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.255959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.255688 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6jmp"] Jul 07 13:17:03.255959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.255804 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 13:17:03.255959 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.255839 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nmbgg\"" Jul 07 13:17:03.256172 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.256062 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 13:17:03.259564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.259544 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwsnb"] Jul 07 13:17:03.287595 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287561 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287616 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287702 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287729 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdpk\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a98509-a7f6-475b-8a44-9f12c809a00c-cert\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.287805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287790 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.288175 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.288175 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.287833 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcmn9\" (UniqueName: \"kubernetes.io/projected/a1a98509-a7f6-475b-8a44-9f12c809a00c-kube-api-access-jcmn9\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.388617 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.388807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.388807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdpk\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.388807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a98509-a7f6-475b-8a44-9f12c809a00c-cert\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.388807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.388807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388808 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d6646f-baa5-4e91-96c0-e614c0f43800-config-volume\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d6646f-baa5-4e91-96c0-e614c0f43800-tmp-dir\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.388971 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.388989 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.388992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcmn9\" (UniqueName: \"kubernetes.io/projected/a1a98509-a7f6-475b-8a44-9f12c809a00c-kube-api-access-jcmn9\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.389063 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:03.889038328 +0000 UTC m=+37.953936260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:03.389104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389536 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389269 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389536 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.389536 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hlc6\" (UniqueName: \"kubernetes.io/projected/d9d6646f-baa5-4e91-96c0-e614c0f43800-kube-api-access-7hlc6\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.389878 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389853 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.390023 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.389999 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.393410 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.393385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.393410 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.393398 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1a98509-a7f6-475b-8a44-9f12c809a00c-cert\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.393582 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.393385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.401242 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.401211 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.401533 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.401510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdpk\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.402020 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.401998 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcmn9\" (UniqueName: \"kubernetes.io/projected/a1a98509-a7f6-475b-8a44-9f12c809a00c-kube-api-access-jcmn9\") pod \"ingress-canary-r6jmp\" (UID: \"a1a98509-a7f6-475b-8a44-9f12c809a00c\") " pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.426898 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.426834 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:03.429756 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.429729 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:17:03.430134 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.429789 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9nksx\"" Jul 07 13:17:03.490187 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d6646f-baa5-4e91-96c0-e614c0f43800-config-volume\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.490322 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d6646f-baa5-4e91-96c0-e614c0f43800-tmp-dir\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.490322 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hlc6\" (UniqueName: \"kubernetes.io/projected/d9d6646f-baa5-4e91-96c0-e614c0f43800-kube-api-access-7hlc6\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.490322 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.490485 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.490405 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:03.490485 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.490468 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:03.990448377 +0000 UTC m=+38.055346308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:03.490598 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490567 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d6646f-baa5-4e91-96c0-e614c0f43800-tmp-dir\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.490776 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.490754 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d6646f-baa5-4e91-96c0-e614c0f43800-config-volume\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.498660 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.498629 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hlc6\" (UniqueName: \"kubernetes.io/projected/d9d6646f-baa5-4e91-96c0-e614c0f43800-kube-api-access-7hlc6\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.560830 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.560795 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6jmp" Jul 07 13:17:03.894237 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.894208 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:03.894383 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.894363 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:03.894423 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.894384 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:03.894456 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.894439 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:04.89442262 +0000 UTC m=+38.959320550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:03.995397 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:03.995285 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:03.995555 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.995416 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:03.995555 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:03.995501 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:04.99547933 +0000 UTC m=+39.060377276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:04.013349 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.013324 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6jmp"] Jul 07 13:17:04.017310 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:04.017284 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1a98509_a7f6_475b_8a44_9f12c809a00c.slice/crio-85cb517b73eccc62f91d3332a7768df7e7d7747b54c92f2cfb4a9b604ddb7bb4 WatchSource:0}: Error finding container 85cb517b73eccc62f91d3332a7768df7e7d7747b54c92f2cfb4a9b604ddb7bb4: Status 404 returned error can't find the container with id 85cb517b73eccc62f91d3332a7768df7e7d7747b54c92f2cfb4a9b604ddb7bb4 Jul 07 13:17:04.427608 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.427387 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:04.427741 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.427384 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:04.430440 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.430410 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:17:04.430842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.430464 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:17:04.431439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.431422 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:17:04.431493 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.431426 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgxmd\"" Jul 07 13:17:04.580002 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.579969 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="a8aadb302b822d91ffb9e244abd68095b089065de4d0495cc69d1191ce31b08c" exitCode=0 Jul 07 13:17:04.580167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.580042 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"a8aadb302b822d91ffb9e244abd68095b089065de4d0495cc69d1191ce31b08c"} Jul 07 13:17:04.581178 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.581159 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6jmp" event={"ID":"a1a98509-a7f6-475b-8a44-9f12c809a00c","Type":"ContainerStarted","Data":"85cb517b73eccc62f91d3332a7768df7e7d7747b54c92f2cfb4a9b604ddb7bb4"} Jul 07 13:17:04.902221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:04.902187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:04.902411 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:04.902358 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:04.902411 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:04.902379 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:04.902532 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:04.902463 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:06.902440778 +0000 UTC m=+40.967338724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:05.002865 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:05.002775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:05.003035 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:05.002948 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:05.003035 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:05.003015 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:07.002999749 +0000 UTC m=+41.067897679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:05.586796 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:05.586762 2573 generic.go:356] "Generic (PLEG): container finished" podID="f038fc69-abff-4174-bfdf-01d488dd6efd" containerID="c1672ae7c26a0a9b17bbbac4f3674ee174dea54fad8c6aa5262e017986f77a14" exitCode=0 Jul 07 13:17:05.587267 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:05.586820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerDied","Data":"c1672ae7c26a0a9b17bbbac4f3674ee174dea54fad8c6aa5262e017986f77a14"} Jul 07 13:17:06.592407 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:06.592371 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" event={"ID":"f038fc69-abff-4174-bfdf-01d488dd6efd","Type":"ContainerStarted","Data":"560f7be855fe0b704893e6602a5301e953bb0ab2e99ca965cb76e42d9cd88b0a"} Jul 07 13:17:06.618155 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:06.615863 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qtd6q" podStartSLOduration=16.256984877 podStartE2EDuration="39.615843289s" totalImagesPullingTime="23.358858412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:06.613783081 +0000 UTC m=+40.678681075" watchObservedRunningTime="2026-07-07 13:17:06.615843289 +0000 UTC m=+40.680741253" Jul 07 13:17:06.918945 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:06.918909 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:06.919029 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:06.919014 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:06.919070 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:06.919031 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:06.919100 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:06.919082 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:10.91906261 +0000 UTC m=+44.983960539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:07.020103 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.020073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:07.020263 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.020245 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:07.020341 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.020298 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:11.020284398 +0000 UTC m=+45.085182332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:07.585702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.585673 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2q4rl"] Jul 07 13:17:07.607006 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.606976 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6jmp" event={"ID":"a1a98509-a7f6-475b-8a44-9f12c809a00c","Type":"ContainerStarted","Data":"e75b95fb9e8e603e94e68771ceb6dc385d432b70dec30a27f81625a777429380"} Jul 07 13:17:07.607006 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.607012 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2q4rl"] Jul 07 13:17:07.607696 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.607093 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.609638 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.609614 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 13:17:07.609761 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.609620 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 13:17:07.609823 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.609781 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nfz7b\"" Jul 07 13:17:07.609823 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.609779 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 13:17:07.609823 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.609786 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 13:17:07.616360 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.616323 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r6jmp" podStartSLOduration=1.7186082219999999 podStartE2EDuration="4.61631131s" totalImagesPullingTime="2.897703088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:07.615575893 +0000 UTC m=+41.680473845" watchObservedRunningTime="2026-07-07 13:17:07.61631131 +0000 UTC m=+41.681209260" Jul 07 13:17:07.626620 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.626599 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b409881a-909e-4fbc-a79e-e448a6eaee5d-data-volume\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.626712 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.626633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.626712 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.626654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.626712 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.626691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b409881a-909e-4fbc-a79e-e448a6eaee5d-crio-socket\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.626834 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.626779 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srxd2\" (UniqueName: \"kubernetes.io/projected/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-api-access-srxd2\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727447 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b409881a-909e-4fbc-a79e-e448a6eaee5d-data-volume\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727594 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727594 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727594 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.727554 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:17:07.727594 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.727557 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:17:07.727751 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.727606 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:08.227588462 +0000 UTC m=+42.292486397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : secret "insights-runtime-extractor-tls" not found Jul 07 13:17:07.727751 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b409881a-909e-4fbc-a79e-e448a6eaee5d-crio-socket\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727751 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:07.727668 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:08.227657773 +0000 UTC m=+42.292555708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : configmap "kube-rbac-proxy" not found Jul 07 13:17:07.727751 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srxd2\" (UniqueName: \"kubernetes.io/projected/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-api-access-srxd2\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727891 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727772 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b409881a-909e-4fbc-a79e-e448a6eaee5d-data-volume\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.727891 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.727827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b409881a-909e-4fbc-a79e-e448a6eaee5d-crio-socket\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:07.745824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:07.745798 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srxd2\" (UniqueName: \"kubernetes.io/projected/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-api-access-srxd2\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:08.230693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.230654 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:08.230693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.230689 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:08.230892 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:08.230777 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 13:17:08.230892 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:08.230793 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:17:08.230892 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:08.230841 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:09.230827984 +0000 UTC m=+43.295725912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : secret "insights-runtime-extractor-tls" not found Jul 07 13:17:08.230892 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:08.230854 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:09.230847377 +0000 UTC m=+43.295745305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : configmap "kube-rbac-proxy" not found Jul 07 13:17:08.537244 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.537164 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr"] Jul 07 13:17:08.572757 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.572721 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr"] Jul 07 13:17:08.572898 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.572830 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" Jul 07 13:17:08.575422 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.575384 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 07 13:17:08.575527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.575430 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-964wk\"" Jul 07 13:17:08.576488 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.576457 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 07 13:17:08.633410 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.633376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzzl8\" (UniqueName: \"kubernetes.io/projected/7577368b-a09b-4d8c-b7fa-e3e52feaed6c-kube-api-access-lzzl8\") pod \"migrator-d57d58cc9-sd8fr\" (UID: \"7577368b-a09b-4d8c-b7fa-e3e52feaed6c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" Jul 07 13:17:08.734071 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.733997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzzl8\" (UniqueName: \"kubernetes.io/projected/7577368b-a09b-4d8c-b7fa-e3e52feaed6c-kube-api-access-lzzl8\") pod \"migrator-d57d58cc9-sd8fr\" (UID: \"7577368b-a09b-4d8c-b7fa-e3e52feaed6c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" Jul 07 13:17:08.742033 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.742005 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzzl8\" (UniqueName: \"kubernetes.io/projected/7577368b-a09b-4d8c-b7fa-e3e52feaed6c-kube-api-access-lzzl8\") pod \"migrator-d57d58cc9-sd8fr\" (UID: \"7577368b-a09b-4d8c-b7fa-e3e52feaed6c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" Jul 07 13:17:08.881541 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:08.881510 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" Jul 07 13:17:09.011968 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.011781 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr"] Jul 07 13:17:09.015336 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:09.015316 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7577368b_a09b_4d8c_b7fa_e3e52feaed6c.slice/crio-b572153a4247236b437318e63f53736eb9c86c88c8a3fecb2524de96ce5848ad WatchSource:0}: Error finding container b572153a4247236b437318e63f53736eb9c86c88c8a3fecb2524de96ce5848ad: Status 404 returned error can't find the container with id b572153a4247236b437318e63f53736eb9c86c88c8a3fecb2524de96ce5848ad Jul 07 13:17:09.237226 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.237115 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:09.237226 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.237153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:09.237406 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:09.237349 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:17:09.237451 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:09.237438 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:11.237415326 +0000 UTC m=+45.302313269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : secret "insights-runtime-extractor-tls" not found Jul 07 13:17:09.237811 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.237791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b409881a-909e-4fbc-a79e-e448a6eaee5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:09.409429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.409401 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fnngb_c313195a-53b5-4a03-a686-d4740f5b112d/dns-node-resolver/0.log" Jul 07 13:17:09.602438 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:09.602361 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" event={"ID":"7577368b-a09b-4d8c-b7fa-e3e52feaed6c","Type":"ContainerStarted","Data":"b572153a4247236b437318e63f53736eb9c86c88c8a3fecb2524de96ce5848ad"} Jul 07 13:17:10.409899 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:10.409870 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zllhn_6977b697-1c8a-4994-a3e0-1430604adb23/node-ca/0.log" Jul 07 13:17:10.949926 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:10.949872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:10.950097 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:10.950027 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:10.950097 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:10.950047 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:10.950172 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:10.950100 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:18.950084301 +0000 UTC m=+53.014982230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:11.010578 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.010535 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6jmp_a1a98509-a7f6-475b-8a44-9f12c809a00c/serve-healthcheck-canary/0.log" Jul 07 13:17:11.051228 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.051200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:11.051366 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:11.051318 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:11.051403 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:11.051368 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:19.051354638 +0000 UTC m=+53.116252568 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:11.253164 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.253096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:11.253297 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:11.253240 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:17:11.253338 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:11.253299 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:15.253283999 +0000 UTC m=+49.318181928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : secret "insights-runtime-extractor-tls" not found Jul 07 13:17:11.608583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.608495 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" event={"ID":"7577368b-a09b-4d8c-b7fa-e3e52feaed6c","Type":"ContainerStarted","Data":"0c0b0cc67c92372e962bfef3124f5c965f07c75b6f7bd254bf236fe8c3579dbe"} Jul 07 13:17:11.608583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.608546 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" event={"ID":"7577368b-a09b-4d8c-b7fa-e3e52feaed6c","Type":"ContainerStarted","Data":"c77c5643b0d57ff288e6d60fbb6e80f8ec5ea1951623fe99d0b4410d5ca282df"} Jul 07 13:17:11.628860 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:11.628814 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sd8fr" podStartSLOduration=1.386385173 podStartE2EDuration="3.628801113s" totalImagesPullingTime="2.24241594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:11.628361368 +0000 UTC m=+45.693259319" watchObservedRunningTime="2026-07-07 13:17:11.628801113 +0000 UTC m=+45.693699060" Jul 07 13:17:15.285772 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:15.285723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:15.286244 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:15.285851 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 13:17:15.286244 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:15.285904 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls podName:b409881a-909e-4fbc-a79e-e448a6eaee5d nodeName:}" failed. No retries permitted until 2026-07-07 13:17:23.28589087 +0000 UTC m=+57.350788799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2q4rl" (UID: "b409881a-909e-4fbc-a79e-e448a6eaee5d") : secret "insights-runtime-extractor-tls" not found Jul 07 13:17:19.015788 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:19.015749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") pod \"image-registry-85cf8895b5-gt29s\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:19.016185 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:19.015904 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 13:17:19.016185 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:19.015935 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85cf8895b5-gt29s: secret "image-registry-tls" not found Jul 07 13:17:19.016185 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:19.015992 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls podName:10c81efa-9bc3-4408-b52f-f506b4d50936 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:35.01597412 +0000 UTC m=+69.080872060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls") pod "image-registry-85cf8895b5-gt29s" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936") : secret "image-registry-tls" not found Jul 07 13:17:19.116621 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:19.116590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:19.116769 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:19.116734 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 13:17:19.116808 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:19.116793 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls podName:d9d6646f-baa5-4e91-96c0-e614c0f43800 nodeName:}" failed. No retries permitted until 2026-07-07 13:17:35.116777893 +0000 UTC m=+69.181675822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls") pod "dns-default-cwsnb" (UID: "d9d6646f-baa5-4e91-96c0-e614c0f43800") : secret "dns-default-metrics-tls" not found Jul 07 13:17:23.346699 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:23.346660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:23.350342 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:23.350323 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b409881a-909e-4fbc-a79e-e448a6eaee5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2q4rl\" (UID: \"b409881a-909e-4fbc-a79e-e448a6eaee5d\") " pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:23.516296 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:23.516264 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2q4rl" Jul 07 13:17:23.640706 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:23.640676 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2q4rl"] Jul 07 13:17:23.643565 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:23.643538 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb409881a_909e_4fbc_a79e_e448a6eaee5d.slice/crio-8992d53900a0b53c91ac41242203471c2096413fb16048ee2fb9956afa099257 WatchSource:0}: Error finding container 8992d53900a0b53c91ac41242203471c2096413fb16048ee2fb9956afa099257: Status 404 returned error can't find the container with id 8992d53900a0b53c91ac41242203471c2096413fb16048ee2fb9956afa099257 Jul 07 13:17:24.640329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:24.640279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q4rl" event={"ID":"b409881a-909e-4fbc-a79e-e448a6eaee5d","Type":"ContainerStarted","Data":"8992d53900a0b53c91ac41242203471c2096413fb16048ee2fb9956afa099257"} Jul 07 13:17:27.580904 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:27.580875 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r2cmc" Jul 07 13:17:28.652949 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:28.652898 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q4rl" event={"ID":"b409881a-909e-4fbc-a79e-e448a6eaee5d","Type":"ContainerStarted","Data":"aa4f252552b246b320edf471c356c19db30dfbf8c7f749753b90a56b7e60dba9"} Jul 07 13:17:29.658136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:29.658097 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q4rl" event={"ID":"b409881a-909e-4fbc-a79e-e448a6eaee5d","Type":"ContainerStarted","Data":"92fe9b647d438308a734ee5d2a62a25398f0c29fe43cced6c6f3fc3bb00961bd"} Jul 07 13:17:31.665042 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:31.665004 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2q4rl" event={"ID":"b409881a-909e-4fbc-a79e-e448a6eaee5d","Type":"ContainerStarted","Data":"deeec392d8eb12ce4bf30c79aa8cfbe79d0f8b7749043e43c1f28461183110a2"} Jul 07 13:17:31.684038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:31.683989 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2q4rl" podStartSLOduration=17.193117875 podStartE2EDuration="24.683975301s" totalImagesPullingTime="7.490857426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:17:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:31.683194969 +0000 UTC m=+65.748092946" watchObservedRunningTime="2026-07-07 13:17:31.683975301 +0000 UTC m=+65.748873230" Jul 07 13:17:32.138190 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.138157 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq"] Jul 07 13:17:32.139897 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.139883 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl"] Jul 07 13:17:32.140029 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.140009 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.141854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.141834 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.146872 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.146828 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 13:17:32.147022 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.146884 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 13:17:32.147022 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.146982 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 13:17:32.147209 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.147192 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 07 13:17:32.147292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.147222 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 07 13:17:32.147477 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.147458 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 07 13:17:32.147578 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.147480 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 07 13:17:32.147578 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.147571 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 13:17:32.157470 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.157449 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl"] Jul 07 13:17:32.158136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.158117 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq"] Jul 07 13:17:32.162877 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.162859 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85cf8895b5-gt29s"] Jul 07 13:17:32.163054 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:17:32.163036 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" podUID="10c81efa-9bc3-4408-b52f-f506b4d50936" Jul 07 13:17:32.218428 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fdxj\" (UniqueName: \"kubernetes.io/projected/28ac697c-88d7-4cfd-a9de-2cce6d61219e-kube-api-access-9fdxj\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218563 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218434 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218563 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218563 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218508 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8884397e-7ccc-4af0-b021-0b8fb5dc7613-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.218563 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8884397e-7ccc-4af0-b021-0b8fb5dc7613-tmp\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.218693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw56w\" (UniqueName: \"kubernetes.io/projected/8884397e-7ccc-4af0-b021-0b8fb5dc7613-kube-api-access-qw56w\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.218693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ac697c-88d7-4cfd-a9de-2cce6d61219e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218693 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.218812 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218720 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:32.218812 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.218742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:32.221238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.221224 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 13:17:32.221348 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.221267 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 13:17:32.231364 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.231344 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7ec2ac7c-f972-4867-a382-81944fe7f93d-original-pull-secret\") pod \"global-pull-secret-syncer-qplbh\" (UID: \"7ec2ac7c-f972-4867-a382-81944fe7f93d\") " pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:32.231469 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.231382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f5f807f-62cd-4965-a59e-952b2e12356c-metrics-certs\") pod \"network-metrics-daemon-28m8c\" (UID: \"6f5f807f-62cd-4965-a59e-952b2e12356c\") " pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:32.238724 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.238706 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9nksx\"" Jul 07 13:17:32.247031 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.247014 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-28m8c" Jul 07 13:17:32.319846 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.319821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8884397e-7ccc-4af0-b021-0b8fb5dc7613-tmp\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.320024 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.319853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw56w\" (UniqueName: \"kubernetes.io/projected/8884397e-7ccc-4af0-b021-0b8fb5dc7613-kube-api-access-qw56w\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.320024 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.319874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320024 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.319891 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ac697c-88d7-4cfd-a9de-2cce6d61219e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320024 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.319990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fdxj\" (UniqueName: \"kubernetes.io/projected/28ac697c-88d7-4cfd-a9de-2cce6d61219e-kube-api-access-9fdxj\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.320238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8884397e-7ccc-4af0-b021-0b8fb5dc7613-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.320238 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:32.320476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320356 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8884397e-7ccc-4af0-b021-0b8fb5dc7613-tmp\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.320974 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.320927 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ac697c-88d7-4cfd-a9de-2cce6d61219e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.323166 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.323145 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.323381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.323363 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 13:17:32.323565 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.323543 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-ca\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.323673 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.323656 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8884397e-7ccc-4af0-b021-0b8fb5dc7613-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.323842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.323821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.324222 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.324205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/28ac697c-88d7-4cfd-a9de-2cce6d61219e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.337279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.337255 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 13:17:32.337515 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.337495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qplbh" Jul 07 13:17:32.339494 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.339471 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fdxj\" (UniqueName: \"kubernetes.io/projected/28ac697c-88d7-4cfd-a9de-2cce6d61219e-kube-api-access-9fdxj\") pod \"cluster-proxy-proxy-agent-7f74c9846c-7dmjl\" (UID: \"28ac697c-88d7-4cfd-a9de-2cce6d61219e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.339734 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.339718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw56w\" (UniqueName: \"kubernetes.io/projected/8884397e-7ccc-4af0-b021-0b8fb5dc7613-kube-api-access-qw56w\") pod \"klusterlet-addon-workmgr-5c7cd9bdbd-plprq\" (UID: \"8884397e-7ccc-4af0-b021-0b8fb5dc7613\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.343764 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.343722 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glct6\" (UniqueName: \"kubernetes.io/projected/d0cd60f0-ab70-4ee8-8e1f-15445dc32906-kube-api-access-glct6\") pod \"network-check-target-dhtsw\" (UID: \"d0cd60f0-ab70-4ee8-8e1f-15445dc32906\") " pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:32.376975 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.376937 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-28m8c"] Jul 07 13:17:32.381000 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:32.380970 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5f807f_62cd_4965_a59e_952b2e12356c.slice/crio-1ba2d9368db35af61656975de6a0eff52939a6e97d35c78269fdc702075536be WatchSource:0}: Error finding container 1ba2d9368db35af61656975de6a0eff52939a6e97d35c78269fdc702075536be: Status 404 returned error can't find the container with id 1ba2d9368db35af61656975de6a0eff52939a6e97d35c78269fdc702075536be Jul 07 13:17:32.451432 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.451401 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:32.466524 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.466496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qplbh"] Jul 07 13:17:32.466664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.466535 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" Jul 07 13:17:32.469396 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:32.469363 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec2ac7c_f972_4867_a382_81944fe7f93d.slice/crio-19f63b71c2ffa8c9f6efcf5ca5c5716d76a0abc685a1d4abc978feee8d9bb12c WatchSource:0}: Error finding container 19f63b71c2ffa8c9f6efcf5ca5c5716d76a0abc685a1d4abc978feee8d9bb12c: Status 404 returned error can't find the container with id 19f63b71c2ffa8c9f6efcf5ca5c5716d76a0abc685a1d4abc978feee8d9bb12c Jul 07 13:17:32.581453 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.581424 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq"] Jul 07 13:17:32.584765 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:32.584736 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8884397e_7ccc_4af0_b021_0b8fb5dc7613.slice/crio-de15c34e7a40409374493310e1e5e39c14ed9c384d4819c19ac94942f929d012 WatchSource:0}: Error finding container de15c34e7a40409374493310e1e5e39c14ed9c384d4819c19ac94942f929d012: Status 404 returned error can't find the container with id de15c34e7a40409374493310e1e5e39c14ed9c384d4819c19ac94942f929d012 Jul 07 13:17:32.598636 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.598609 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl"] Jul 07 13:17:32.601314 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:32.601291 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28ac697c_88d7_4cfd_a9de_2cce6d61219e.slice/crio-63738aa5bb977175e454008153caa5ec87620eddcdcff32a213dc416a434b525 WatchSource:0}: Error finding container 63738aa5bb977175e454008153caa5ec87620eddcdcff32a213dc416a434b525: Status 404 returned error can't find the container with id 63738aa5bb977175e454008153caa5ec87620eddcdcff32a213dc416a434b525 Jul 07 13:17:32.644873 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.644814 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgxmd\"" Jul 07 13:17:32.653055 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.653036 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:32.669505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.669475 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28m8c" event={"ID":"6f5f807f-62cd-4965-a59e-952b2e12356c","Type":"ContainerStarted","Data":"1ba2d9368db35af61656975de6a0eff52939a6e97d35c78269fdc702075536be"} Jul 07 13:17:32.670577 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.670554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" event={"ID":"28ac697c-88d7-4cfd-a9de-2cce6d61219e","Type":"ContainerStarted","Data":"63738aa5bb977175e454008153caa5ec87620eddcdcff32a213dc416a434b525"} Jul 07 13:17:32.671721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.671697 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qplbh" event={"ID":"7ec2ac7c-f972-4867-a382-81944fe7f93d","Type":"ContainerStarted","Data":"19f63b71c2ffa8c9f6efcf5ca5c5716d76a0abc685a1d4abc978feee8d9bb12c"} Jul 07 13:17:32.672852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.672825 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:32.672852 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.672835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" event={"ID":"8884397e-7ccc-4af0-b021-0b8fb5dc7613","Type":"ContainerStarted","Data":"de15c34e7a40409374493310e1e5e39c14ed9c384d4819c19ac94942f929d012"} Jul 07 13:17:32.678958 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.678940 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:32.724184 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724158 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724208 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724248 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724283 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724309 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724522 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724445 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.724522 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.724509 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vdpk\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk\") pod \"10c81efa-9bc3-4408-b52f-f506b4d50936\" (UID: \"10c81efa-9bc3-4408-b52f-f506b4d50936\") " Jul 07 13:17:32.725323 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.725272 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:32.725452 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.725409 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:17:32.725556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.725542 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:17:32.726650 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.726616 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:32.727385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.727362 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:17:32.727470 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.727365 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk" (OuterVolumeSpecName: "kube-api-access-2vdpk") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "kube-api-access-2vdpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:17:32.727641 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.727624 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "10c81efa-9bc3-4408-b52f-f506b4d50936" (UID: "10c81efa-9bc3-4408-b52f-f506b4d50936"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:17:32.775673 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.775643 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dhtsw"] Jul 07 13:17:32.778747 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:32.778712 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cd60f0_ab70_4ee8_8e1f_15445dc32906.slice/crio-c5c8e993e12227fe97500a3a9817f059e56988de9e714664b9f196aae62fdf3a WatchSource:0}: Error finding container c5c8e993e12227fe97500a3a9817f059e56988de9e714664b9f196aae62fdf3a: Status 404 returned error can't find the container with id c5c8e993e12227fe97500a3a9817f059e56988de9e714664b9f196aae62fdf3a Jul 07 13:17:32.826188 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826156 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-trusted-ca\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826188 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826186 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-certificates\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826197 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-image-registry-private-configuration\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826206 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-bound-sa-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826214 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/10c81efa-9bc3-4408-b52f-f506b4d50936-ca-trust-extracted\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826222 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vdpk\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-kube-api-access-2vdpk\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:32.826390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:32.826236 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/10c81efa-9bc3-4408-b52f-f506b4d50936-installation-pull-secrets\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:33.679462 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.679424 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28m8c" event={"ID":"6f5f807f-62cd-4965-a59e-952b2e12356c","Type":"ContainerStarted","Data":"98fe12ed9516ef09d1ecb068dce07101b19212ed4377234b7e218833cbb49da2"} Jul 07 13:17:33.690502 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.687407 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85cf8895b5-gt29s" Jul 07 13:17:33.695723 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.695172 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhtsw" event={"ID":"d0cd60f0-ab70-4ee8-8e1f-15445dc32906","Type":"ContainerStarted","Data":"c5c8e993e12227fe97500a3a9817f059e56988de9e714664b9f196aae62fdf3a"} Jul 07 13:17:33.742934 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.742825 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85cf8895b5-gt29s"] Jul 07 13:17:33.749216 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.749165 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85cf8895b5-gt29s"] Jul 07 13:17:33.835260 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:33.835230 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/10c81efa-9bc3-4408-b52f-f506b4d50936-registry-tls\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:17:34.433797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:34.433585 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c81efa-9bc3-4408-b52f-f506b4d50936" path="/var/lib/kubelet/pods/10c81efa-9bc3-4408-b52f-f506b4d50936/volumes" Jul 07 13:17:34.701180 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:34.701095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-28m8c" event={"ID":"6f5f807f-62cd-4965-a59e-952b2e12356c","Type":"ContainerStarted","Data":"0c897276de54782736f641236fd41e80c636571343fe2c63cf20e6ce4be15362"} Jul 07 13:17:34.721437 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:34.720793 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-28m8c" podStartSLOduration=66.561869033 podStartE2EDuration="1m7.720774068s" totalImagesPullingTime="1.158905035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:34.71878141 +0000 UTC m=+68.783679384" watchObservedRunningTime="2026-07-07 13:17:34.720774068 +0000 UTC m=+68.785672020" Jul 07 13:17:35.145996 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:35.145530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:35.150118 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:35.149790 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9d6646f-baa5-4e91-96c0-e614c0f43800-metrics-tls\") pod \"dns-default-cwsnb\" (UID: \"d9d6646f-baa5-4e91-96c0-e614c0f43800\") " pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:35.369807 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:35.369607 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nmbgg\"" Jul 07 13:17:35.377927 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:35.377627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:36.412993 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:36.412833 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cwsnb"] Jul 07 13:17:39.913385 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:17:39.913353 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d6646f_baa5_4e91_96c0_e614c0f43800.slice/crio-dfbaf725659acba3ffef77f68e7cfd4ec0e0ffb5d083e22598ced2316bc388b0 WatchSource:0}: Error finding container dfbaf725659acba3ffef77f68e7cfd4ec0e0ffb5d083e22598ced2316bc388b0: Status 404 returned error can't find the container with id dfbaf725659acba3ffef77f68e7cfd4ec0e0ffb5d083e22598ced2316bc388b0 Jul 07 13:17:40.728366 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:40.728322 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwsnb" event={"ID":"d9d6646f-baa5-4e91-96c0-e614c0f43800","Type":"ContainerStarted","Data":"dfbaf725659acba3ffef77f68e7cfd4ec0e0ffb5d083e22598ced2316bc388b0"} Jul 07 13:17:41.734668 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.734574 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" event={"ID":"28ac697c-88d7-4cfd-a9de-2cce6d61219e","Type":"ContainerStarted","Data":"57e64fcbb982bb096c43696e46918db3b693d2ea7aa41a9d2c784d4ba765a8ae"} Jul 07 13:17:41.734668 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.734622 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" event={"ID":"28ac697c-88d7-4cfd-a9de-2cce6d61219e","Type":"ContainerStarted","Data":"58f1fcdc9aea3a2df9ded02e2490c7a7fe872644bdf3fdcc3207ac3306cd2033"} Jul 07 13:17:41.734668 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.734642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" event={"ID":"28ac697c-88d7-4cfd-a9de-2cce6d61219e","Type":"ContainerStarted","Data":"f9a34bfe31922146e42dcb93253c5e36347af79bba67f6539369208c56b0e54a"} Jul 07 13:17:41.736672 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.736615 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qplbh" event={"ID":"7ec2ac7c-f972-4867-a382-81944fe7f93d","Type":"ContainerStarted","Data":"73968a89dbe29c0307b872ddb26c23d9fb048037add0408c047a05e752e1967a"} Jul 07 13:17:41.738397 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.738315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" event={"ID":"8884397e-7ccc-4af0-b021-0b8fb5dc7613","Type":"ContainerStarted","Data":"8660f6e4940e979f5150904cc86aaba25c0fe9dcc48a3f50b3fd73b1e5286782"} Jul 07 13:17:41.738518 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.738447 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:41.740176 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.740152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dhtsw" event={"ID":"d0cd60f0-ab70-4ee8-8e1f-15445dc32906","Type":"ContainerStarted","Data":"ccb8e46f4ef7411422ccda1b3135f266c3bc0906e82da5974c01a484ba06653b"} Jul 07 13:17:41.740282 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.740209 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:17:41.740381 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.740363 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" Jul 07 13:17:41.753052 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.752998 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7f74c9846c-7dmjl" podStartSLOduration=0.988458759 podStartE2EDuration="9.752985653s" totalImagesPullingTime="8.764526894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:41.751357527 +0000 UTC m=+75.816255470" watchObservedRunningTime="2026-07-07 13:17:41.752985653 +0000 UTC m=+75.817883603" Jul 07 13:17:41.767023 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.766970 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c7cd9bdbd-plprq" podStartSLOduration=0.981301153 podStartE2EDuration="9.766952786s" totalImagesPullingTime="8.785651633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:41.766723228 +0000 UTC m=+75.831621180" watchObservedRunningTime="2026-07-07 13:17:41.766952786 +0000 UTC m=+75.831850739" Jul 07 13:17:41.785027 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.784963 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dhtsw" podStartSLOduration=66.198798016 podStartE2EDuration="1m14.784949338s" totalImagesPullingTime="8.586151322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:41.783177351 +0000 UTC m=+75.848075303" watchObservedRunningTime="2026-07-07 13:17:41.784949338 +0000 UTC m=+75.849847288" Jul 07 13:17:41.801805 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:41.801762 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qplbh" podStartSLOduration=65.912950415 podStartE2EDuration="1m14.801747822s" totalImagesPullingTime="8.888797407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:41.800901354 +0000 UTC m=+75.865799305" watchObservedRunningTime="2026-07-07 13:17:41.801747822 +0000 UTC m=+75.866645750" Jul 07 13:17:43.748153 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:43.748113 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwsnb" event={"ID":"d9d6646f-baa5-4e91-96c0-e614c0f43800","Type":"ContainerStarted","Data":"f2d1dbdda10495d7aea4732f868b8e6b3829c729ceeaaf07f3200fa73781a974"} Jul 07 13:17:43.748615 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:43.748183 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cwsnb" event={"ID":"d9d6646f-baa5-4e91-96c0-e614c0f43800","Type":"ContainerStarted","Data":"334e5f27f26f0d6c55de53da02cb2cf2293f5f4a87ea9df7c448467ad8b2cab9"} Jul 07 13:17:43.765603 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:43.765557 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cwsnb" podStartSLOduration=37.993186353 podStartE2EDuration="40.7655414s" totalImagesPullingTime="2.772355047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:17:43.763937627 +0000 UTC m=+77.828835569" watchObservedRunningTime="2026-07-07 13:17:43.7655414 +0000 UTC m=+77.830439351" Jul 07 13:17:44.751637 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:44.751607 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cwsnb" Jul 07 13:17:54.757402 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:17:54.757277 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cwsnb" Jul 07 13:18:01.601979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.601945 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-94nsf"] Jul 07 13:18:01.615882 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.615859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.618439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.618416 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 13:18:01.618753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.618735 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 13:18:01.618845 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.618769 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 13:18:01.618903 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.618850 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wjwb8\"" Jul 07 13:18:01.618903 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.618882 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 13:18:01.619950 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.619936 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 13:18:01.620003 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.619965 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 13:18:01.736033 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.735991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736033 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736028 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-textfile\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736052 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-metrics-client-ca\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736086 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-root\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-accelerators-collector-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736137 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-sys\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r92rw\" (UniqueName: \"kubernetes.io/projected/7a43f530-62f7-4a51-9084-4c8e1f364796-kube-api-access-r92rw\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.736261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.736241 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-wtmp\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.836907 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.836870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-textfile\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.836907 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.836905 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.836944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-metrics-client-ca\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.836969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-root\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837012 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-accelerators-collector-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837038 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-root\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-sys\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:18:01.837045 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r92rw\" (UniqueName: \"kubernetes.io/projected/7a43f530-62f7-4a51-9084-4c8e1f364796-kube-api-access-r92rw\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-wtmp\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:18:01.837128 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls podName:7a43f530-62f7-4a51-9084-4c8e1f364796 nodeName:}" failed. No retries permitted until 2026-07-07 13:18:02.33710508 +0000 UTC m=+96.402003014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls") pod "node-exporter-94nsf" (UID: "7a43f530-62f7-4a51-9084-4c8e1f364796") : secret "node-exporter-tls" not found Jul 07 13:18:01.837167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837157 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-sys\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837214 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-wtmp\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837546 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-textfile\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837637 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837588 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-metrics-client-ca\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.837685 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.837666 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-accelerators-collector-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.839362 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.839347 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:01.846176 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:01.846151 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r92rw\" (UniqueName: \"kubernetes.io/projected/7a43f530-62f7-4a51-9084-4c8e1f364796-kube-api-access-r92rw\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:02.343249 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:02.343204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:02.345494 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:02.345474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a43f530-62f7-4a51-9084-4c8e1f364796-node-exporter-tls\") pod \"node-exporter-94nsf\" (UID: \"7a43f530-62f7-4a51-9084-4c8e1f364796\") " pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:02.524980 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:02.524945 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-94nsf" Jul 07 13:18:02.534642 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:18:02.534614 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a43f530_62f7_4a51_9084_4c8e1f364796.slice/crio-473e90d15687c1cec8a47b770b6ce557e3c0d592b19c759bdb1658d243a680bd WatchSource:0}: Error finding container 473e90d15687c1cec8a47b770b6ce557e3c0d592b19c759bdb1658d243a680bd: Status 404 returned error can't find the container with id 473e90d15687c1cec8a47b770b6ce557e3c0d592b19c759bdb1658d243a680bd Jul 07 13:18:02.808232 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:02.808193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94nsf" event={"ID":"7a43f530-62f7-4a51-9084-4c8e1f364796","Type":"ContainerStarted","Data":"473e90d15687c1cec8a47b770b6ce557e3c0d592b19c759bdb1658d243a680bd"} Jul 07 13:18:03.813739 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:03.813709 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94nsf" event={"ID":"7a43f530-62f7-4a51-9084-4c8e1f364796","Type":"ContainerStarted","Data":"bf88511780bf700f523df459e3a7dca3131924ac56224e154d0844736952688b"} Jul 07 13:18:04.818869 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:04.818831 2573 generic.go:356] "Generic (PLEG): container finished" podID="7a43f530-62f7-4a51-9084-4c8e1f364796" containerID="bf88511780bf700f523df459e3a7dca3131924ac56224e154d0844736952688b" exitCode=0 Jul 07 13:18:04.819268 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:04.818904 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94nsf" event={"ID":"7a43f530-62f7-4a51-9084-4c8e1f364796","Type":"ContainerDied","Data":"bf88511780bf700f523df459e3a7dca3131924ac56224e154d0844736952688b"} Jul 07 13:18:05.824071 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:05.824034 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94nsf" event={"ID":"7a43f530-62f7-4a51-9084-4c8e1f364796","Type":"ContainerStarted","Data":"19dec48e643917aadb8676c2fb3d2b49cd403ee5f2a8d8c3c6d58e113d45b92a"} Jul 07 13:18:05.824437 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:05.824095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-94nsf" event={"ID":"7a43f530-62f7-4a51-9084-4c8e1f364796","Type":"ContainerStarted","Data":"3d90950ab1109273baf3d4a33646c814469e4680af52da9fd3a209bbb7e1f4dc"} Jul 07 13:18:05.853090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:05.850425 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-94nsf" podStartSLOduration=3.592617899 podStartE2EDuration="4.850403255s" totalImagesPullingTime="1.187192386s" totalInitContainerRuntime="70.59297ms" isStatefulPod=true podCreationTimestamp="2026-07-07 13:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:18:05.848113913 +0000 UTC m=+99.913011899" watchObservedRunningTime="2026-07-07 13:18:05.850403255 +0000 UTC m=+99.915301210" Jul 07 13:18:12.746279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:12.746248 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dhtsw" Jul 07 13:18:13.978952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:18:13.978877 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6jmp_a1a98509-a7f6-475b-8a44-9f12c809a00c/serve-healthcheck-canary/0.log" Jul 07 13:20:59.030393 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.030304 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg"] Jul 07 13:20:59.032336 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.032315 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.034905 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.034884 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 13:20:59.035062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.034892 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7rw22\"" Jul 07 13:20:59.035847 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.035833 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 13:20:59.040063 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.040041 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg"] Jul 07 13:20:59.125732 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.125696 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jsbm\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-kube-api-access-4jsbm\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.125901 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.125739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.227123 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.227089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jsbm\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-kube-api-access-4jsbm\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.227288 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.227141 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.235568 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.235545 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.235839 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.235816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jsbm\" (UniqueName: \"kubernetes.io/projected/ba03ec50-0203-43a6-8e04-0c24cf076862-kube-api-access-4jsbm\") pod \"cert-manager-cainjector-559cdd8c87-n4mbg\" (UID: \"ba03ec50-0203-43a6-8e04-0c24cf076862\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.341052 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.340974 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" Jul 07 13:20:59.465647 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:20:59.465613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg"] Jul 07 13:20:59.469756 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:20:59.469729 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba03ec50_0203_43a6_8e04_0c24cf076862.slice/crio-21e2b7b6d36581a6af1a281af7184d874d27bb261841a46109c79d5ae383c7b2 WatchSource:0}: Error finding container 21e2b7b6d36581a6af1a281af7184d874d27bb261841a46109c79d5ae383c7b2: Status 404 returned error can't find the container with id 21e2b7b6d36581a6af1a281af7184d874d27bb261841a46109c79d5ae383c7b2 Jul 07 13:21:00.385441 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:00.385400 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" event={"ID":"ba03ec50-0203-43a6-8e04-0c24cf076862","Type":"ContainerStarted","Data":"21e2b7b6d36581a6af1a281af7184d874d27bb261841a46109c79d5ae383c7b2"} Jul 07 13:21:02.397564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:02.397516 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" event={"ID":"ba03ec50-0203-43a6-8e04-0c24cf076862","Type":"ContainerStarted","Data":"0e53d12719413c0de58c4a43f6459b82b75fad40ff4f61301093a3ff8fcc7ddc"} Jul 07 13:21:02.413987 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:02.413937 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n4mbg" podStartSLOduration=0.69989221 podStartE2EDuration="3.413907108s" totalImagesPullingTime="2.714014898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:02.413122469 +0000 UTC m=+276.478020422" watchObservedRunningTime="2026-07-07 13:21:02.413907108 +0000 UTC m=+276.478805052" Jul 07 13:21:26.303682 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:26.303656 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 13:21:27.141274 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.141245 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s"] Jul 07 13:21:27.143694 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.143676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.147695 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147671 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 13:21:27.147695 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147675 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 13:21:27.147876 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147671 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 13:21:27.147876 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147733 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 13:21:27.147876 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147767 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kl9xm\"" Jul 07 13:21:27.147876 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.147854 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 13:21:27.153349 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.153331 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s"] Jul 07 13:21:27.229784 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.229754 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-manager-config\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.230038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.229801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.230038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.229825 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-metrics-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.230038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.229876 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8s6\" (UniqueName: \"kubernetes.io/projected/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-kube-api-access-cq8s6\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.330458 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.330423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8s6\" (UniqueName: \"kubernetes.io/projected/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-kube-api-access-cq8s6\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.330824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.330476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-manager-config\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.330824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.330523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.330824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.330552 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-metrics-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.331183 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.331166 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-manager-config\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.332961 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.332941 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-metrics-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.333071 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.332995 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-cert\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.340244 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.340221 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8s6\" (UniqueName: \"kubernetes.io/projected/1742be8f-79c5-4b71-aa00-2ab6f7ab55ab-kube-api-access-cq8s6\") pod \"lws-controller-manager-76d5cc9db8-p5m4s\" (UID: \"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab\") " pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.452948 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.452846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:27.580286 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.580257 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s"] Jul 07 13:21:27.584378 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:21:27.584356 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1742be8f_79c5_4b71_aa00_2ab6f7ab55ab.slice/crio-7a041cb02a30896d1d30fc5ee67b4a0d602d0288611931bffdbaf6a1b5a00b23 WatchSource:0}: Error finding container 7a041cb02a30896d1d30fc5ee67b4a0d602d0288611931bffdbaf6a1b5a00b23: Status 404 returned error can't find the container with id 7a041cb02a30896d1d30fc5ee67b4a0d602d0288611931bffdbaf6a1b5a00b23 Jul 07 13:21:27.586371 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:27.586355 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:21:28.478785 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:28.478752 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" event={"ID":"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab","Type":"ContainerStarted","Data":"7a041cb02a30896d1d30fc5ee67b4a0d602d0288611931bffdbaf6a1b5a00b23"} Jul 07 13:21:30.487163 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:30.487126 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" event={"ID":"1742be8f-79c5-4b71-aa00-2ab6f7ab55ab","Type":"ContainerStarted","Data":"b7692adeac3f0a68be03102e08f376d28c6a3f2d848589662f4d53c1e5a4e524"} Jul 07 13:21:30.487656 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:30.487262 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:30.504203 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:30.504158 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" podStartSLOduration=1.236737424 podStartE2EDuration="3.504144589s" totalImagesPullingTime="2.267407165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:30.503465338 +0000 UTC m=+304.568363324" watchObservedRunningTime="2026-07-07 13:21:30.504144589 +0000 UTC m=+304.569042539" Jul 07 13:21:33.701481 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.701444 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v"] Jul 07 13:21:33.704638 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.704622 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.708226 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.708200 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 13:21:33.708364 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.708254 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 13:21:33.708364 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.708338 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4vc28\"" Jul 07 13:21:33.708463 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.708444 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 13:21:33.708987 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.708966 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 13:21:33.723716 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.723686 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v"] Jul 07 13:21:33.882574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.882536 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-tmp\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.882574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.882577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-manifests\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.882793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.882611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.882793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.882655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.882793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.882669 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz4z9\" (UniqueName: \"kubernetes.io/projected/b26ba531-5055-44df-a4a0-b935043ae01c-kube-api-access-hz4z9\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983382 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-tmp\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983382 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-manifests\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983382 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983554 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983554 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz4z9\" (UniqueName: \"kubernetes.io/projected/b26ba531-5055-44df-a4a0-b935043ae01c-kube-api-access-hz4z9\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.983856 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.983825 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-manifests\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.985702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.985680 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b26ba531-5055-44df-a4a0-b935043ae01c-tmp\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.985801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.985775 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.985801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.985784 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b26ba531-5055-44df-a4a0-b935043ae01c-webhook-cert\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:33.993060 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:33.993041 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz4z9\" (UniqueName: \"kubernetes.io/projected/b26ba531-5055-44df-a4a0-b935043ae01c-kube-api-access-hz4z9\") pod \"opendatahub-operator-controller-manager-69fbc88fbc-t7k4v\" (UID: \"b26ba531-5055-44df-a4a0-b935043ae01c\") " pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:34.014661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:34.014634 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:34.153626 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:34.153598 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v"] Jul 07 13:21:34.156991 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:21:34.156965 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26ba531_5055_44df_a4a0_b935043ae01c.slice/crio-a13fe195f91a03e5592c2b3e70f6ae8e86527cab902ab4e244651aa6cb9f4684 WatchSource:0}: Error finding container a13fe195f91a03e5592c2b3e70f6ae8e86527cab902ab4e244651aa6cb9f4684: Status 404 returned error can't find the container with id a13fe195f91a03e5592c2b3e70f6ae8e86527cab902ab4e244651aa6cb9f4684 Jul 07 13:21:34.500994 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:34.500958 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" event={"ID":"b26ba531-5055-44df-a4a0-b935043ae01c","Type":"ContainerStarted","Data":"a13fe195f91a03e5592c2b3e70f6ae8e86527cab902ab4e244651aa6cb9f4684"} Jul 07 13:21:37.514439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:37.514402 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" event={"ID":"b26ba531-5055-44df-a4a0-b935043ae01c","Type":"ContainerStarted","Data":"e4f68136aa2c7d60e490c0340cb49e9aa1c885a10147a67c056c1cd4211f7859"} Jul 07 13:21:38.519147 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:38.519107 2573 generic.go:356] "Generic (PLEG): container finished" podID="b26ba531-5055-44df-a4a0-b935043ae01c" containerID="e4f68136aa2c7d60e490c0340cb49e9aa1c885a10147a67c056c1cd4211f7859" exitCode=0 Jul 07 13:21:38.519525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:38.519156 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" event={"ID":"b26ba531-5055-44df-a4a0-b935043ae01c","Type":"ContainerDied","Data":"e4f68136aa2c7d60e490c0340cb49e9aa1c885a10147a67c056c1cd4211f7859"} Jul 07 13:21:39.525860 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:39.525821 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" event={"ID":"b26ba531-5055-44df-a4a0-b935043ae01c","Type":"ContainerStarted","Data":"3936216c48aea018ab7b4b55439ceff3a9ce580f14d508976324b5ba19ac7291"} Jul 07 13:21:39.526302 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:39.526040 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:21:39.548227 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:39.548178 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" podStartSLOduration=1.927902808 podStartE2EDuration="6.548161963s" totalImagesPullingTime="3.577876663s" totalInitContainerRuntime="1.042382492s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:21:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:21:39.545824497 +0000 UTC m=+313.610722451" watchObservedRunningTime="2026-07-07 13:21:39.548161963 +0000 UTC m=+313.613059914" Jul 07 13:21:41.493651 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:41.493615 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76d5cc9db8-p5m4s" Jul 07 13:21:50.533715 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:21:50.533684 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69fbc88fbc-t7k4v" Jul 07 13:22:01.150837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.150802 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk"] Jul 07 13:22:01.163196 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.163171 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.165932 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.165895 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 13:22:01.165932 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.165897 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 13:22:01.166111 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.165986 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p9sph\"" Jul 07 13:22:01.167187 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.167166 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 13:22:01.168787 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.168755 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk"] Jul 07 13:22:01.243680 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.243651 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr"] Jul 07 13:22:01.246735 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.246719 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.249405 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.249383 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 13:22:01.249513 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.249386 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 13:22:01.249513 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.249484 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-78lwn\"" Jul 07 13:22:01.249624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.249610 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 13:22:01.262866 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.262839 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr"] Jul 07 13:22:01.282274 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282398 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282287 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282398 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282398 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282362 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282398 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282390 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7zd\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-kube-api-access-8m7zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282438 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6pj9\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-kube-api-access-v6pj9\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffd934a-f74a-4ddb-ac3b-8daeba401117-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282491 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec299975-b4bc-4557-b54a-618346342c5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.282729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282722 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.282819 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.282741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec299975-b4bc-4557-b54a-618346342c5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.383933 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.383890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec299975-b4bc-4557-b54a-618346342c5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.383945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384045 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384218 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384069 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384218 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384218 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec299975-b4bc-4557-b54a-618346342c5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384370 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384235 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384370 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384269 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384370 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384297 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384370 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7zd\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-kube-api-access-8m7zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384422 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6pj9\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-kube-api-access-v6pj9\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384425 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384462 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffd934a-f74a-4ddb-ac3b-8daeba401117-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384467 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384485 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.384989 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384989 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384883 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec299975-b4bc-4557-b54a-618346342c5f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.384989 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.384900 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.385644 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.385579 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.386885 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.386857 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec299975-b4bc-4557-b54a-618346342c5f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.387094 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.387071 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.387268 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.387248 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec299975-b4bc-4557-b54a-618346342c5f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.387344 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.387303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cffd934a-f74a-4ddb-ac3b-8daeba401117-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.387423 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.387407 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.387582 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.387568 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cffd934a-f74a-4ddb-ac3b-8daeba401117-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.394053 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.394030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7zd\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-kube-api-access-8m7zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.394320 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.394299 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6pj9\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-kube-api-access-v6pj9\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.394404 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.394331 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cffd934a-f74a-4ddb-ac3b-8daeba401117-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-58rqr\" (UID: \"cffd934a-f74a-4ddb-ac3b-8daeba401117\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.394544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.394524 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec299975-b4bc-4557-b54a-618346342c5f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h9kdk\" (UID: \"ec299975-b4bc-4557-b54a-618346342c5f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.474263 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.474182 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:01.556742 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.556714 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:01.658175 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.658152 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk"] Jul 07 13:22:01.662172 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:22:01.662133 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec299975_b4bc_4557_b54a_618346342c5f.slice/crio-2c9478f6ecae39670726ccdd52e37a67cf9d082148ae7d6f313417376ed9ee2a WatchSource:0}: Error finding container 2c9478f6ecae39670726ccdd52e37a67cf9d082148ae7d6f313417376ed9ee2a: Status 404 returned error can't find the container with id 2c9478f6ecae39670726ccdd52e37a67cf9d082148ae7d6f313417376ed9ee2a Jul 07 13:22:01.738632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:01.738607 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr"] Jul 07 13:22:01.741056 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:22:01.741031 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffd934a_f74a_4ddb_ac3b_8daeba401117.slice/crio-1f0f6b599f36aeccebb1dfdc881167da398fcd81ee641399a808c433a58f5d2b WatchSource:0}: Error finding container 1f0f6b599f36aeccebb1dfdc881167da398fcd81ee641399a808c433a58f5d2b: Status 404 returned error can't find the container with id 1f0f6b599f36aeccebb1dfdc881167da398fcd81ee641399a808c433a58f5d2b Jul 07 13:22:02.613679 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:02.613641 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" event={"ID":"ec299975-b4bc-4557-b54a-618346342c5f","Type":"ContainerStarted","Data":"2c9478f6ecae39670726ccdd52e37a67cf9d082148ae7d6f313417376ed9ee2a"} Jul 07 13:22:02.615347 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:02.615320 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" event={"ID":"cffd934a-f74a-4ddb-ac3b-8daeba401117","Type":"ContainerStarted","Data":"1f0f6b599f36aeccebb1dfdc881167da398fcd81ee641399a808c433a58f5d2b"} Jul 07 13:22:05.394561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.394518 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:22:05.394945 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.394603 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:22:05.398495 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.398468 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:22:05.398580 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.398547 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:22:05.398645 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.398604 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:22:05.630246 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.629999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" event={"ID":"ec299975-b4bc-4557-b54a-618346342c5f","Type":"ContainerStarted","Data":"d6fea548a79fb6eab7e5118ae687c7b71602666bbc5a51012ad07a6e5f559489"} Jul 07 13:22:05.631865 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.631836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" event={"ID":"cffd934a-f74a-4ddb-ac3b-8daeba401117","Type":"ContainerStarted","Data":"578f759d373fd45c24858c5321a1f2f0accfbcf1da4b8cb29bded4e39ba0c7b4"} Jul 07 13:22:05.632034 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.631909 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:22:05.668775 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.668669 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" podStartSLOduration=0.934492435 podStartE2EDuration="4.668654688s" totalImagesPullingTime="3.734162253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:05.666827787 +0000 UTC m=+339.731725738" watchObservedRunningTime="2026-07-07 13:22:05.668654688 +0000 UTC m=+339.733552638" Jul 07 13:22:05.719085 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:05.719034 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" podStartSLOduration=1.067750749 podStartE2EDuration="4.719016926s" totalImagesPullingTime="3.651266177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:22:05.718770619 +0000 UTC m=+339.783668571" watchObservedRunningTime="2026-07-07 13:22:05.719016926 +0000 UTC m=+339.783914876" Jul 07 13:22:06.475121 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:06.475093 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:06.479479 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:06.479454 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:06.635870 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:06.635827 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:06.636600 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:06.636578 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h9kdk" Jul 07 13:22:06.637240 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:22:06.637222 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-58rqr" Jul 07 13:24:33.049796 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.049761 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:24:33.052750 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.052734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.055509 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.055489 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 13:24:33.055616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.055490 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 13:24:33.056527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.056512 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ms98c\"" Jul 07 13:24:33.056579 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.056547 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 13:24:33.060398 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.060377 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:24:33.090110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.090085 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:24:33.180746 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.180713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dk44\" (UniqueName: \"kubernetes.io/projected/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-kube-api-access-5dk44\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.180935 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.180775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-config-file\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.281316 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.281277 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-config-file\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.281480 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.281347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dk44\" (UniqueName: \"kubernetes.io/projected/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-kube-api-access-5dk44\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.282048 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.282029 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-config-file\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.291237 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.291212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dk44\" (UniqueName: \"kubernetes.io/projected/834bf5a8-f347-4ed5-8831-dab1e5ea8bca-kube-api-access-5dk44\") pod \"limitador-limitador-54d6d988d5-sfnsw\" (UID: \"834bf5a8-f347-4ed5-8831-dab1e5ea8bca\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.363236 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.363202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:33.489905 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:33.489876 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:24:33.493121 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:24:33.493093 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834bf5a8_f347_4ed5_8831_dab1e5ea8bca.slice/crio-477f3a890d8b16786d7339508d415f5883651a8874c29d8cd88de2a847e6af5e WatchSource:0}: Error finding container 477f3a890d8b16786d7339508d415f5883651a8874c29d8cd88de2a847e6af5e: Status 404 returned error can't find the container with id 477f3a890d8b16786d7339508d415f5883651a8874c29d8cd88de2a847e6af5e Jul 07 13:24:34.159757 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:34.159716 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" event={"ID":"834bf5a8-f347-4ed5-8831-dab1e5ea8bca","Type":"ContainerStarted","Data":"477f3a890d8b16786d7339508d415f5883651a8874c29d8cd88de2a847e6af5e"} Jul 07 13:24:36.169850 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:36.169812 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" event={"ID":"834bf5a8-f347-4ed5-8831-dab1e5ea8bca","Type":"ContainerStarted","Data":"f39fb9d01a7423a2b217cc95b1b3ff5d2bac5b2cde397a9ae6baa90a18d8a45e"} Jul 07 13:24:36.170252 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:36.169948 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:24:36.185768 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:36.185721 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" podStartSLOduration=0.730347192 podStartE2EDuration="3.185706022s" totalImagesPullingTime="2.45535883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:24:36.184234143 +0000 UTC m=+490.249132191" watchObservedRunningTime="2026-07-07 13:24:36.185706022 +0000 UTC m=+490.250603974" Jul 07 13:24:47.174700 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:24:47.174670 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sfnsw" Jul 07 13:25:45.404258 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:25:45.404217 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:26:09.790049 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:26:09.790015 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:26:28.080813 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:26:28.080777 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:26:54.489639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:26:54.489560 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:27:01.678324 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.678284 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4"] Jul 07 13:27:01.681376 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.681359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.684218 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.684199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 13:27:01.685304 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.685283 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 13:27:01.685388 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.685335 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 13:27:01.685443 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.685386 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qgt7b\"" Jul 07 13:27:01.691711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.691691 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4"] Jul 07 13:27:01.754427 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.754603 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754543 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.754603 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.754732 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754654 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.754732 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjl7b\" (UniqueName: \"kubernetes.io/projected/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-kube-api-access-fjl7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.754822 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.754737 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.855861 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.855822 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856058 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.855873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856058 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.855978 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjl7b\" (UniqueName: \"kubernetes.io/projected/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-kube-api-access-fjl7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856058 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856058 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856260 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856334 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856401 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856340 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.856401 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.856394 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.858134 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.858110 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.858436 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.858421 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.863404 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.863383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjl7b\" (UniqueName: \"kubernetes.io/projected/c038e44f-154f-4a68-977f-f6d0bbcaa1c8-kube-api-access-fjl7b\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4\" (UID: \"c038e44f-154f-4a68-977f-f6d0bbcaa1c8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:01.992351 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:01.992269 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:02.117348 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:02.117318 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4"] Jul 07 13:27:02.120963 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:27:02.120937 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc038e44f_154f_4a68_977f_f6d0bbcaa1c8.slice/crio-7e40ea7c7082993130ff6badb0efa50abc25a89fd5c3d3374aa93c1bd62fe09c WatchSource:0}: Error finding container 7e40ea7c7082993130ff6badb0efa50abc25a89fd5c3d3374aa93c1bd62fe09c: Status 404 returned error can't find the container with id 7e40ea7c7082993130ff6badb0efa50abc25a89fd5c3d3374aa93c1bd62fe09c Jul 07 13:27:02.122226 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:02.122208 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 13:27:02.696876 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:02.696835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" event={"ID":"c038e44f-154f-4a68-977f-f6d0bbcaa1c8","Type":"ContainerStarted","Data":"7e40ea7c7082993130ff6badb0efa50abc25a89fd5c3d3374aa93c1bd62fe09c"} Jul 07 13:27:03.188261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:03.187618 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:27:03.706847 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:03.706802 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" event={"ID":"c038e44f-154f-4a68-977f-f6d0bbcaa1c8","Type":"ContainerStarted","Data":"fef656ca515bd260e00cbc17b091a64185600c312b779d13b1d5e5f743d9d76a"} Jul 07 13:27:03.723834 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:03.723785 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" podStartSLOduration=1.601944999 podStartE2EDuration="2.723770837s" totalImagesPullingTime="1.121825838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:27:03.722476808 +0000 UTC m=+637.787374761" watchObservedRunningTime="2026-07-07 13:27:03.723770837 +0000 UTC m=+637.788668787" Jul 07 13:27:11.993394 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:11.993346 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:11.993783 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:11.993424 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:12.006000 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:12.005973 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:12.752195 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:12.752168 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4" Jul 07 13:27:16.280466 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:16.280430 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:27:30.784010 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:27:30.783975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:29:38.589219 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:29:38.589183 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:29:49.483478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:29:49.483396 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:30:00.143589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.143554 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-ww4sd"] Jul 07 13:30:00.146787 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.146771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:00.149574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.149555 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9r9sk\"" Jul 07 13:30:00.155121 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.155097 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-ww4sd"] Jul 07 13:30:00.273719 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.273684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfsg9\" (UniqueName: \"kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9\") pod \"maas-api-key-cleanup-29723850-ww4sd\" (UID: \"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7\") " pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:00.374216 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.374172 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfsg9\" (UniqueName: \"kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9\") pod \"maas-api-key-cleanup-29723850-ww4sd\" (UID: \"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7\") " pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:00.383040 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.383015 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfsg9\" (UniqueName: \"kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9\") pod \"maas-api-key-cleanup-29723850-ww4sd\" (UID: \"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7\") " pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:00.457887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.457816 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:00.585005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.584976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723850-ww4sd"] Jul 07 13:30:00.587875 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:30:00.587850 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd5a256_9b9d_4a89_beca_5c8ea6f1edc7.slice/crio-bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a WatchSource:0}: Error finding container bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a: Status 404 returned error can't find the container with id bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a Jul 07 13:30:00.791534 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:00.791454 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:30:01.358761 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:01.358729 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" event={"ID":"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7","Type":"ContainerStarted","Data":"bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a"} Jul 07 13:30:04.372584 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:04.372544 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" event={"ID":"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7","Type":"ContainerStarted","Data":"464a1ee9371ce4e0068c0097b6b664e3297c9aa8fd61016f847d010816c15513"} Jul 07 13:30:04.389042 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:04.388996 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" podStartSLOduration=1.329107152 podStartE2EDuration="4.388981238s" totalImagesPullingTime="3.059874086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:30:04.386978898 +0000 UTC m=+818.451876879" watchObservedRunningTime="2026-07-07 13:30:04.388981238 +0000 UTC m=+818.453879188" Jul 07 13:30:05.377764 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:05.377730 2573 generic.go:356] "Generic (PLEG): container finished" podID="ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7" containerID="464a1ee9371ce4e0068c0097b6b664e3297c9aa8fd61016f847d010816c15513" exitCode=0 Jul 07 13:30:05.378155 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:05.377769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" event={"ID":"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7","Type":"ContainerDied","Data":"464a1ee9371ce4e0068c0097b6b664e3297c9aa8fd61016f847d010816c15513"} Jul 07 13:30:06.511766 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:06.511743 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:06.623820 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:06.623788 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfsg9\" (UniqueName: \"kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9\") pod \"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7\" (UID: \"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7\") " Jul 07 13:30:06.625991 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:06.625952 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9" (OuterVolumeSpecName: "kube-api-access-hfsg9") pod "ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7" (UID: "ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7"). InnerVolumeSpecName "kube-api-access-hfsg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:30:06.724558 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:06.724522 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfsg9\" (UniqueName: \"kubernetes.io/projected/ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7-kube-api-access-hfsg9\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:30:07.388691 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:07.388664 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" Jul 07 13:30:07.388691 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:07.388672 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723850-ww4sd" event={"ID":"ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7","Type":"ContainerDied","Data":"bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a"} Jul 07 13:30:07.388894 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:07.388713 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc7a8007c2704a84f1c7882d3c66ce534a9135a7fc1992768b0ef8f6ae72028a" Jul 07 13:30:09.283197 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:09.283163 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:30:19.691905 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:19.691872 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:30:45.989258 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:45.989219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:30:58.186930 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:30:58.186892 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:31:31.884115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:31:31.884080 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:31:40.576442 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:31:40.576407 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:31:50.576909 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:31:50.576869 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:31:58.879565 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:31:58.879529 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:32:11.397997 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:32:11.397968 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:32:27.284943 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:32:27.284891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:32:33.691653 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:32:33.691615 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:32:41.984566 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:32:41.984531 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:33:04.476677 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:33:04.476598 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:33:16.581886 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:33:16.581855 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:33:38.685697 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:33:38.685660 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:33:49.479289 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:33:49.479260 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:34:05.682321 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:34:05.682290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:34:16.794004 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:34:16.793970 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:34:26.191729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:34:26.191638 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:34:34.489652 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:34:34.489613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:34:43.085371 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:34:43.085334 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:36:34.899446 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:36:34.899412 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:36:45.888096 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:36:45.888057 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:36:54.303718 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:36:54.303684 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:04.301573 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:04.301540 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:15.090756 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:15.090727 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:26.696592 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:26.696509 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:36.180262 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:36.180231 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:46.085088 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:46.085054 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:37:55.988117 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:37:55.988080 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:03.589286 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:03.589252 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:15.086843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:15.086806 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:22.391853 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:22.391823 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:40.096814 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:40.096776 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:49.386052 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:49.385976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:38:59.387455 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:38:59.387420 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:11.183314 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:11.183281 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:18.294720 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:18.294686 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:27.780897 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:27.780862 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:40.891005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:40.890974 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:49.587191 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:49.587156 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:39:59.089016 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:39:59.088985 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:40:08.096859 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:40:08.096821 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:40:16.886975 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:40:16.886942 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:40:29.486293 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:40:29.486217 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:40:39.294131 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:40:39.294091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:40:50.391302 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:40:50.391266 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:00.294385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:00.294347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:12.194369 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:12.194339 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:19.785856 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:19.785823 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:29.789084 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:29.789041 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:40.794405 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:40.794372 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:41:50.985262 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:41:50.985168 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:03.390027 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:03.389994 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:11.785140 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:11.785103 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:21.183784 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:21.183742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:22.022434 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.022405 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:22.022703 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.022691 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7" containerName="cleanup" Jul 07 13:42:22.022757 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.022706 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7" containerName="cleanup" Jul 07 13:42:22.022790 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.022755 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7" containerName="cleanup" Jul 07 13:42:22.025826 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.025803 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.028794 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.028770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0fbb7bc5-openshift-default-dockercfg-rgxrr\"" Jul 07 13:42:22.049089 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.049069 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:22.113019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.112987 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nggx9\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113477 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113219 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113477 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.113477 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.113300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.213977 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214013 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nggx9\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214126 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214229 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214262 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214509 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214875 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214589 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214875 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214676 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214875 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.214875 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.214763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.216612 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.216592 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.216731 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.216716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.222476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.222453 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nggx9\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.223258 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.223240 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token\") pod \"e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.336705 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.336638 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:22.511771 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:42:22.511692 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b80299_d8f3_4d03_a14d_9df54c7405b2.slice/crio-3f7ed52dd92cd5a02cbe4668cdc9fc5d64fb3a279a5beaa20bb52100bb3ec873 WatchSource:0}: Error finding container 3f7ed52dd92cd5a02cbe4668cdc9fc5d64fb3a279a5beaa20bb52100bb3ec873: Status 404 returned error can't find the container with id 3f7ed52dd92cd5a02cbe4668cdc9fc5d64fb3a279a5beaa20bb52100bb3ec873 Jul 07 13:42:22.513771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.513742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:22.514019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.513984 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:22.514128 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.514069 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:22.514128 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:22.514116 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:23.301275 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:23.301237 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" event={"ID":"51b80299-d8f3-4d03-a14d-9df54c7405b2","Type":"ContainerStarted","Data":"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d"} Jul 07 13:42:23.301733 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:23.301308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" event={"ID":"51b80299-d8f3-4d03-a14d-9df54c7405b2","Type":"ContainerStarted","Data":"3f7ed52dd92cd5a02cbe4668cdc9fc5d64fb3a279a5beaa20bb52100bb3ec873"} Jul 07 13:42:23.335694 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:23.335475 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" podStartSLOduration=2.335455874 podStartE2EDuration="2.335455874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:23.335179374 +0000 UTC m=+1557.400077325" watchObservedRunningTime="2026-07-07 13:42:23.335455874 +0000 UTC m=+1557.400353825" Jul 07 13:42:23.337292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:23.337267 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:23.341655 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:23.341635 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:24.305296 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:24.305262 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:24.306165 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:24.306147 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:28.510509 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:28.510474 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:28.511009 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:28.510669 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" podUID="51b80299-d8f3-4d03-a14d-9df54c7405b2" containerName="istio-proxy" containerID="cri-o://257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d" gracePeriod=30 Jul 07 13:42:33.549682 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.549655 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:33.617133 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617108 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617166 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nggx9\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617186 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617219 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617238 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617254 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617292 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617274 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617600 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617298 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617600 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617324 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert\") pod \"51b80299-d8f3-4d03-a14d-9df54c7405b2\" (UID: \"51b80299-d8f3-4d03-a14d-9df54c7405b2\") " Jul 07 13:42:33.617600 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617480 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:33.617751 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617607 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:33.617751 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617613 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.617854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617783 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:42:33.617854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.617803 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:33.618194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.618168 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:33.619664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.619632 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:33.619771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.619712 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:33.619771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.619727 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:42:33.619771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.619745 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9" (OuterVolumeSpecName: "kube-api-access-nggx9") pod "51b80299-d8f3-4d03-a14d-9df54c7405b2" (UID: "51b80299-d8f3-4d03-a14d-9df54c7405b2"). InnerVolumeSpecName "kube-api-access-nggx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:33.718313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718275 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718306 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nggx9\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-kube-api-access-nggx9\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718316 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718325 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718333 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718342 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718350 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51b80299-d8f3-4d03-a14d-9df54c7405b2-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:33.718574 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:33.718358 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51b80299-d8f3-4d03-a14d-9df54c7405b2-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:34.354588 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.354553 2573 generic.go:356] "Generic (PLEG): container finished" podID="51b80299-d8f3-4d03-a14d-9df54c7405b2" containerID="257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d" exitCode=0 Jul 07 13:42:34.354776 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.354608 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" Jul 07 13:42:34.354776 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.354610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" event={"ID":"51b80299-d8f3-4d03-a14d-9df54c7405b2","Type":"ContainerDied","Data":"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d"} Jul 07 13:42:34.354776 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.354740 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm" event={"ID":"51b80299-d8f3-4d03-a14d-9df54c7405b2","Type":"ContainerDied","Data":"3f7ed52dd92cd5a02cbe4668cdc9fc5d64fb3a279a5beaa20bb52100bb3ec873"} Jul 07 13:42:34.354983 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.354783 2573 scope.go:117] "RemoveContainer" containerID="257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d" Jul 07 13:42:34.363450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.363433 2573 scope.go:117] "RemoveContainer" containerID="257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d" Jul 07 13:42:34.363689 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:42:34.363667 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d\": container with ID starting with 257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d not found: ID does not exist" containerID="257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d" Jul 07 13:42:34.363735 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.363698 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d"} err="failed to get container status \"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d\": rpc error: code = NotFound desc = could not find container \"257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d\": container with ID starting with 257671fe2d06d7ca713d8b54ac528b13ae4685d5d21738a1f034c17bca9c919d not found: ID does not exist" Jul 07 13:42:34.376634 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.376603 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:34.380978 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.380958 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0fbb7bc5-openshift-default-749b447cc7-pf7nm"] Jul 07 13:42:34.431823 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.431795 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b80299-d8f3-4d03-a14d-9df54c7405b2" path="/var/lib/kubelet/pods/51b80299-d8f3-4d03-a14d-9df54c7405b2/volumes" Jul 07 13:42:34.536198 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.536170 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:34.536509 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.536492 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b80299-d8f3-4d03-a14d-9df54c7405b2" containerName="istio-proxy" Jul 07 13:42:34.536589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.536512 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b80299-d8f3-4d03-a14d-9df54c7405b2" containerName="istio-proxy" Jul 07 13:42:34.536589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.536586 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b80299-d8f3-4d03-a14d-9df54c7405b2" containerName="istio-proxy" Jul 07 13:42:34.539445 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.539421 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.542229 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.542205 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ae77d6a7-openshift-default-dockercfg-rpmlk\"" Jul 07 13:42:34.554339 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.554318 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:34.624767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.624767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624769 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.624987 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.624987 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.624987 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.625079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.624987 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.625079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.625017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.625079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.625043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.625170 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.625076 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98cf4\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726246 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98cf4\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726425 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726408 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726503 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726563 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726675 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726889 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726721 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.726889 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726809 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.727036 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.726984 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.727257 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.727233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.728834 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.728807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.728988 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.728968 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.735969 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.735950 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98cf4\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.736190 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.736174 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token\") pod \"e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.849721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.849686 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:34.982815 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.982783 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:34.985148 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:42:34.985121 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4beac88b_a0ef_4796_a89c_0b45367987cf.slice/crio-b7d84855b2a981466d641854809ffa3133a62109f15495f99292c2e280c22f6a WatchSource:0}: Error finding container b7d84855b2a981466d641854809ffa3133a62109f15495f99292c2e280c22f6a: Status 404 returned error can't find the container with id b7d84855b2a981466d641854809ffa3133a62109f15495f99292c2e280c22f6a Jul 07 13:42:34.987366 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.987332 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:34.987466 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.987403 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:34.987466 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:34.987453 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:35.361583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:35.361557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerStarted","Data":"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f"} Jul 07 13:42:35.361728 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:35.361605 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerStarted","Data":"b7d84855b2a981466d641854809ffa3133a62109f15495f99292c2e280c22f6a"} Jul 07 13:42:35.385479 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:35.385428 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" podStartSLOduration=1.385411796 podStartE2EDuration="1.385411796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:35.385048368 +0000 UTC m=+1569.449946356" watchObservedRunningTime="2026-07-07 13:42:35.385411796 +0000 UTC m=+1569.450309747" Jul 07 13:42:35.850684 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:35.850594 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:36.854897 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:36.854868 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:37.369336 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:37.369307 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:37.370408 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:37.370386 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:38.191173 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:38.191098 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:46.412890 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.412860 2573 generic.go:356] "Generic (PLEG): container finished" podID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerID="fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f" exitCode=0 Jul 07 13:42:46.413313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.412930 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerDied","Data":"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f"} Jul 07 13:42:46.413313 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.413266 2573 scope.go:117] "RemoveContainer" containerID="fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f" Jul 07 13:42:46.414215 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.414187 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:46.414271 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.414259 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:46.414306 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:46.414290 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:47.010342 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:47.010307 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:47.418311 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:47.418277 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerStarted","Data":"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642"} Jul 07 13:42:47.418723 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:47.418349 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" containerID="cri-o://a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642" gracePeriod=30 Jul 07 13:42:49.281456 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:49.281419 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:52.369965 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.369908 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:52.681067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.681044 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:52.783842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.783813 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.783866 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.783894 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.783940 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.783971 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784026 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784058 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784084 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98cf4\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784119 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert\") pod \"4beac88b-a0ef-4796-a89c-0b45367987cf\" (UID: \"4beac88b-a0ef-4796-a89c-0b45367987cf\") " Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784145 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:52.784290 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784223 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:52.784583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784322 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:52.784583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784429 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.784583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784448 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.784583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784463 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.784778 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.784750 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:42:52.785212 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.785191 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:52.786161 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.786136 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:42:52.786272 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.786233 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4" (OuterVolumeSpecName: "kube-api-access-98cf4") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "kube-api-access-98cf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:52.786504 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.786484 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:42:52.786563 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.786504 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "4beac88b-a0ef-4796-a89c-0b45367987cf" (UID: "4beac88b-a0ef-4796-a89c-0b45367987cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:42:52.885750 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885709 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.885750 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885744 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.885750 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885753 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.885750 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885761 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4beac88b-a0ef-4796-a89c-0b45367987cf-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.886004 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885769 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98cf4\" (UniqueName: \"kubernetes.io/projected/4beac88b-a0ef-4796-a89c-0b45367987cf-kube-api-access-98cf4\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:52.886004 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:52.885777 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4beac88b-a0ef-4796-a89c-0b45367987cf-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:42:53.311714 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.311679 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:42:53.312015 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.311999 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.312064 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.312018 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.312064 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.312036 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.312064 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.312042 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.312149 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.312093 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.312149 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.312102 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerName="istio-proxy" Jul 07 13:42:53.314167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.314149 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.317021 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.316998 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a4c66a8f-openshift-default-dockercfg-8xhtj\"" Jul 07 13:42:53.327992 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.327969 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:42:53.390642 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390609 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.390642 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390700 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46l9v\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390806 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.391105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.390896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.444588 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.444555 2573 generic.go:356] "Generic (PLEG): container finished" podID="4beac88b-a0ef-4796-a89c-0b45367987cf" containerID="a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642" exitCode=0 Jul 07 13:42:53.444744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.444615 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" Jul 07 13:42:53.444744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.444635 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerDied","Data":"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642"} Jul 07 13:42:53.444744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.444687 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s" event={"ID":"4beac88b-a0ef-4796-a89c-0b45367987cf","Type":"ContainerDied","Data":"b7d84855b2a981466d641854809ffa3133a62109f15495f99292c2e280c22f6a"} Jul 07 13:42:53.444744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.444710 2573 scope.go:117] "RemoveContainer" containerID="a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642" Jul 07 13:42:53.453554 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.453538 2573 scope.go:117] "RemoveContainer" containerID="fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f" Jul 07 13:42:53.460501 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.460485 2573 scope.go:117] "RemoveContainer" containerID="a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642" Jul 07 13:42:53.460735 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:42:53.460718 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642\": container with ID starting with a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642 not found: ID does not exist" containerID="a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642" Jul 07 13:42:53.460813 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.460747 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642"} err="failed to get container status \"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642\": rpc error: code = NotFound desc = could not find container \"a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642\": container with ID starting with a679e3f5dcad53a5c37b0ca09b6b988edef044a3fbdf985b5a9a051f9dea2642 not found: ID does not exist" Jul 07 13:42:53.460813 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.460770 2573 scope.go:117] "RemoveContainer" containerID="fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f" Jul 07 13:42:53.461040 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:42:53.461022 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f\": container with ID starting with fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f not found: ID does not exist" containerID="fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f" Jul 07 13:42:53.461094 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.461047 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f"} err="failed to get container status \"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f\": rpc error: code = NotFound desc = could not find container \"fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f\": container with ID starting with fe907c28ef3c64e5f4b13567c53eacfe92636450733774363f1ca6eb548e608f not found: ID does not exist" Jul 07 13:42:53.466003 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.465981 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:53.470411 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.470392 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ae77d6a7-openshift-default-7999474b99-jbv4s"] Jul 07 13:42:53.492227 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492208 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46l9v\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492238 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492301 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492345 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492680 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492736 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492786 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492768 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492854 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.492946 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.492910 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.493104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.493083 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.494647 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.494621 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.494928 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.494896 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.503682 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.503656 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.503851 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.503835 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46l9v\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v\") pod \"e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.625181 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.625151 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:53.756721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.756630 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:42:53.758966 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:42:53.758937 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd4cd4c_7d89_41b6_8b4e_4086b4153640.slice/crio-a7ad30c92ff2905ef3c05d95d32c9aec388b08d05f974045140544a861623da7 WatchSource:0}: Error finding container a7ad30c92ff2905ef3c05d95d32c9aec388b08d05f974045140544a861623da7: Status 404 returned error can't find the container with id a7ad30c92ff2905ef3c05d95d32c9aec388b08d05f974045140544a861623da7 Jul 07 13:42:53.760926 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.760879 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:53.761018 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.760969 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:53.761018 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:53.760996 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:42:54.433655 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.433622 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4beac88b-a0ef-4796-a89c-0b45367987cf" path="/var/lib/kubelet/pods/4beac88b-a0ef-4796-a89c-0b45367987cf/volumes" Jul 07 13:42:54.451434 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.451400 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" event={"ID":"7dd4cd4c-7d89-41b6-8b4e-4086b4153640","Type":"ContainerStarted","Data":"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b"} Jul 07 13:42:54.451598 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.451443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" event={"ID":"7dd4cd4c-7d89-41b6-8b4e-4086b4153640","Type":"ContainerStarted","Data":"a7ad30c92ff2905ef3c05d95d32c9aec388b08d05f974045140544a861623da7"} Jul 07 13:42:54.476899 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.476854 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" podStartSLOduration=1.476839269 podStartE2EDuration="1.476839269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:42:54.474693937 +0000 UTC m=+1588.539591909" watchObservedRunningTime="2026-07-07 13:42:54.476839269 +0000 UTC m=+1588.541737220" Jul 07 13:42:54.625762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.625731 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:54.643828 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:54.643784 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:55.460646 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:55.458147 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:55.461858 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:55.461833 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:42:57.487153 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:57.487120 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:42:59.146988 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:59.146952 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:42:59.147481 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:42:59.147223 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" podUID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" containerName="istio-proxy" containerID="cri-o://4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b" gracePeriod=30 Jul 07 13:43:05.190030 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.190006 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:43:05.299079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.298996 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299030 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46l9v\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299055 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299079 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299076 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299103 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299153 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299183 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299209 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299239 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token\") pod \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\" (UID: \"7dd4cd4c-7d89-41b6-8b4e-4086b4153640\") " Jul 07 13:43:05.299450 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299387 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:43:05.299740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299480 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:43:05.299740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299504 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.299740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299589 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:43:05.299740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.299723 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:43:05.300094 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.300062 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data" (OuterVolumeSpecName: "istio-data") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:43:05.301387 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.301347 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v" (OuterVolumeSpecName: "kube-api-access-46l9v") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "kube-api-access-46l9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:43:05.301387 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.301375 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token" (OuterVolumeSpecName: "istio-token") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:43:05.301544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.301414 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:43:05.301544 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.301424 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7dd4cd4c-7d89-41b6-8b4e-4086b4153640" (UID: "7dd4cd4c-7d89-41b6-8b4e-4086b4153640"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:43:05.399877 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399841 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.399877 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399872 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46l9v\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-kube-api-access-46l9v\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.399877 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399883 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.400115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399892 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.400115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399900 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.400115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399908 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.400115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399934 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.400115 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.399943 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7dd4cd4c-7d89-41b6-8b4e-4086b4153640-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:43:05.499886 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.499851 2573 generic.go:356] "Generic (PLEG): container finished" podID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" containerID="4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b" exitCode=0 Jul 07 13:43:05.500062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.499898 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" event={"ID":"7dd4cd4c-7d89-41b6-8b4e-4086b4153640","Type":"ContainerDied","Data":"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b"} Jul 07 13:43:05.500062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.499960 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" event={"ID":"7dd4cd4c-7d89-41b6-8b4e-4086b4153640","Type":"ContainerDied","Data":"a7ad30c92ff2905ef3c05d95d32c9aec388b08d05f974045140544a861623da7"} Jul 07 13:43:05.500062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.499981 2573 scope.go:117] "RemoveContainer" containerID="4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b" Jul 07 13:43:05.500062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.499900 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9" Jul 07 13:43:05.508883 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.508866 2573 scope.go:117] "RemoveContainer" containerID="4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b" Jul 07 13:43:05.509131 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:43:05.509111 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b\": container with ID starting with 4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b not found: ID does not exist" containerID="4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b" Jul 07 13:43:05.509176 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.509139 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b"} err="failed to get container status \"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b\": rpc error: code = NotFound desc = could not find container \"4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b\": container with ID starting with 4530757816a3472a3cc4f9d7b687e58989a0791cce20c599b11ce06ae43c8e6b not found: ID does not exist" Jul 07 13:43:05.521014 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.520989 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:43:05.526363 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.526342 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a4c66a8f-openshift-default-947b5679-q59n9"] Jul 07 13:43:05.987815 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:05.987774 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:43:06.432875 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:43:06.432837 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" path="/var/lib/kubelet/pods/7dd4cd4c-7d89-41b6-8b4e-4086b4153640/volumes" Jul 07 13:44:27.391358 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:27.391323 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:44:34.481783 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:34.481747 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:44:41.109718 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.109681 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:44:41.110136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.109981 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" containerName="istio-proxy" Jul 07 13:44:41.110136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.109994 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" containerName="istio-proxy" Jul 07 13:44:41.110136 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.110045 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dd4cd4c-7d89-41b6-8b4e-4086b4153640" containerName="istio-proxy" Jul 07 13:44:41.112885 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.112869 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.115732 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.115710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ba6ae1a8-openshift-default-dockercfg-g5wsg\"" Jul 07 13:44:41.129365 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.129340 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:44:41.228872 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.228839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229054 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.228879 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229054 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.228907 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229054 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229034 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229156 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229064 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229156 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229085 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229222 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229170 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8dlb\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229222 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.229283 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.229221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.329955 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.329925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.329965 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.329984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8dlb\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330037 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330106 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330091 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330152 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330519 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330490 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330607 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330582 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330665 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330643 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330713 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330670 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.330969 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.330952 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.332170 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.332149 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.332403 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.332384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.337733 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.337710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.337733 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.337716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8dlb\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb\") pod \"e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.423343 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.423308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:41.557818 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.557707 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:44:41.560032 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:44:41.560003 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246e25b4_8c35_481d_a596_c765c86c5646.slice/crio-2d7974e9beb22e6863f14890b7e01675feecf78ade5f1751a0b41728bd1510a4 WatchSource:0}: Error finding container 2d7974e9beb22e6863f14890b7e01675feecf78ade5f1751a0b41728bd1510a4: Status 404 returned error can't find the container with id 2d7974e9beb22e6863f14890b7e01675feecf78ade5f1751a0b41728bd1510a4 Jul 07 13:44:41.562105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.562076 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:41.562196 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.562137 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:41.562196 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.562166 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:41.883171 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.883131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerStarted","Data":"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45"} Jul 07 13:44:41.883344 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.883188 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerStarted","Data":"2d7974e9beb22e6863f14890b7e01675feecf78ade5f1751a0b41728bd1510a4"} Jul 07 13:44:41.904394 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:41.904347 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podStartSLOduration=0.904332884 podStartE2EDuration="904.332884ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:44:41.902667572 +0000 UTC m=+1695.967565525" watchObservedRunningTime="2026-07-07 13:44:41.904332884 +0000 UTC m=+1695.969230834" Jul 07 13:44:42.424474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:42.424425 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:42.432588 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:42.432564 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:42.887182 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:42.887153 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:42.888040 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:42.888020 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:42.980048 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:42.980019 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:44:52.077378 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:52.077304 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:44:56.949841 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.949807 2573 generic.go:356] "Generic (PLEG): container finished" podID="246e25b4-8c35-481d-a596-c765c86c5646" containerID="85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45" exitCode=0 Jul 07 13:44:56.950247 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.949875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerDied","Data":"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45"} Jul 07 13:44:56.950288 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.950257 2573 scope.go:117] "RemoveContainer" containerID="85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45" Jul 07 13:44:56.951235 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.951210 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:56.951291 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.951278 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:56.951327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:56.951310 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:44:57.887101 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:57.887068 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:57.955507 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:57.955471 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerStarted","Data":"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce"} Jul 07 13:44:58.423864 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:58.423770 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:44:58.425554 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:58.425523 2573 patch_prober.go:28] interesting pod/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 07 13:44:58.425857 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:58.425817 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 07 13:44:58.790105 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:58.790013 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:44:59.423759 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:59.423717 2573 patch_prober.go:28] interesting pod/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 07 13:44:59.424385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:44:59.424349 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 07 13:45:00.423801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:00.423750 2573 patch_prober.go:28] interesting pod/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 07 13:45:00.424327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:00.423838 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 07 13:45:01.424611 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:01.424567 2573 patch_prober.go:28] interesting pod/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 07 13:45:01.425239 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:01.425199 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 07 13:45:02.432621 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:02.432594 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:45:02.432994 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:02.432937 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:45:02.433648 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:02.433631 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:45:02.476555 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:02.476526 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:45:02.975685 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:02.975657 2573 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" secret="" err="secret \"e2e-mt-ba6ae1a8-openshift-default-dockercfg-g5wsg\" not found" Jul 07 13:45:03.979790 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:03.979755 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" containerID="cri-o://9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce" gracePeriod=30 Jul 07 13:45:09.230591 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.230564 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:45:09.278382 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278294 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278382 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278342 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278451 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278481 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278511 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278558 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278817 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278618 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.278817 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278663 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:45:09.278817 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278797 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:45:09.278977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278811 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:45:09.278977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278856 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/246e25b4-8c35-481d-a596-c765c86c5646-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.278977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.278964 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:45:09.279252 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.279230 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data" (OuterVolumeSpecName: "istio-data") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:45:09.280603 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.280579 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:45:09.280773 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.280750 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:45:09.379602 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379564 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8dlb\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.379797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379610 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token\") pod \"246e25b4-8c35-481d-a596-c765c86c5646\" (UID: \"246e25b4-8c35-481d-a596-c765c86c5646\") " Jul 07 13:45:09.379862 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379815 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/246e25b4-8c35-481d-a596-c765c86c5646-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.379862 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379833 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.379862 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379847 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.379862 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379860 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.380044 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379873 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.380044 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.379887 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/246e25b4-8c35-481d-a596-c765c86c5646-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.381735 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.381712 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token" (OuterVolumeSpecName: "istio-token") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:45:09.381735 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.381723 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb" (OuterVolumeSpecName: "kube-api-access-w8dlb") pod "246e25b4-8c35-481d-a596-c765c86c5646" (UID: "246e25b4-8c35-481d-a596-c765c86c5646"). InnerVolumeSpecName "kube-api-access-w8dlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:45:09.481062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.481015 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8dlb\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-kube-api-access-w8dlb\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:09.481062 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:09.481056 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/246e25b4-8c35-481d-a596-c765c86c5646-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:45:10.007131 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.007100 2573 generic.go:356] "Generic (PLEG): container finished" podID="246e25b4-8c35-481d-a596-c765c86c5646" containerID="9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce" exitCode=0 Jul 07 13:45:10.007294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.007143 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" Jul 07 13:45:10.007294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.007182 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerDied","Data":"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce"} Jul 07 13:45:10.007294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.007245 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh" event={"ID":"246e25b4-8c35-481d-a596-c765c86c5646","Type":"ContainerDied","Data":"2d7974e9beb22e6863f14890b7e01675feecf78ade5f1751a0b41728bd1510a4"} Jul 07 13:45:10.007294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.007274 2573 scope.go:117] "RemoveContainer" containerID="9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce" Jul 07 13:45:10.015890 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.015872 2573 scope.go:117] "RemoveContainer" containerID="85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45" Jul 07 13:45:10.023024 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.023004 2573 scope.go:117] "RemoveContainer" containerID="9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce" Jul 07 13:45:10.023272 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:45:10.023255 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce\": container with ID starting with 9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce not found: ID does not exist" containerID="9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce" Jul 07 13:45:10.023319 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.023284 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce"} err="failed to get container status \"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce\": rpc error: code = NotFound desc = could not find container \"9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce\": container with ID starting with 9134ee668af294da4743951af79a88e1c2fa6074e1c30380e4fd7508791469ce not found: ID does not exist" Jul 07 13:45:10.023319 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.023301 2573 scope.go:117] "RemoveContainer" containerID="85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45" Jul 07 13:45:10.023539 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:45:10.023523 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45\": container with ID starting with 85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45 not found: ID does not exist" containerID="85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45" Jul 07 13:45:10.023573 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.023548 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45"} err="failed to get container status \"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45\": rpc error: code = NotFound desc = could not find container \"85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45\": container with ID starting with 85c1602b68279206df6ee23a4cb38fc82049ff79d23c2a6e471f162dcf169e45 not found: ID does not exist" Jul 07 13:45:10.028401 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.028379 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:45:10.030633 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.030612 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ba6ae1a8-openshift-default-859766c879-5dmgh"] Jul 07 13:45:10.431189 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:10.431155 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="246e25b4-8c35-481d-a596-c765c86c5646" path="/var/lib/kubelet/pods/246e25b4-8c35-481d-a596-c765c86c5646/volumes" Jul 07 13:45:22.307418 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.307380 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:45:22.307935 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.307885 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.307935 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.307907 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.308067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.307952 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.308067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.307962 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.308067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.308028 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.308067 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.308039 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="246e25b4-8c35-481d-a596-c765c86c5646" containerName="istio-proxy" Jul 07 13:45:22.312581 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.312557 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.315192 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.315171 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e1603ba5-openshift-default-dockercfg-wgqfk\"" Jul 07 13:45:22.321556 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.321530 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:45:22.384253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384253 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384253 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384336 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384474 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384665 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rbn9\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.384665 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.384509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485224 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rbn9\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485224 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485246 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485767 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485980 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485895 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.485980 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485967 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.486110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.485996 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.486110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.486039 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.486175 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.486161 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.488088 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.488062 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.488402 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.488380 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.493439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.493418 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.493713 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.493692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rbn9\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9\") pod \"e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.623903 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.623867 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:22.762392 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.762369 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:45:22.764888 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:45:22.764848 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d3e46d_bfa9_445d_b42c_1742fd62e15b.slice/crio-7fec97ee734e994130edefadc900e6df970267d4d44fea5163c6ad90c2b1e668 WatchSource:0}: Error finding container 7fec97ee734e994130edefadc900e6df970267d4d44fea5163c6ad90c2b1e668: Status 404 returned error can't find the container with id 7fec97ee734e994130edefadc900e6df970267d4d44fea5163c6ad90c2b1e668 Jul 07 13:45:22.766887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.766858 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:22.766977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.766939 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:22.766977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:22.766973 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:23.062606 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:23.062507 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerStarted","Data":"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591"} Jul 07 13:45:23.062606 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:23.062571 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerStarted","Data":"7fec97ee734e994130edefadc900e6df970267d4d44fea5163c6ad90c2b1e668"} Jul 07 13:45:23.086390 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:23.086319 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podStartSLOduration=1.086299995 podStartE2EDuration="1.086299995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:45:23.083325412 +0000 UTC m=+1737.148223363" watchObservedRunningTime="2026-07-07 13:45:23.086299995 +0000 UTC m=+1737.151197947" Jul 07 13:45:23.624467 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:23.624424 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:23.629900 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:23.629874 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:24.066485 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:24.066400 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:24.067576 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:24.067556 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:29.285284 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:29.285251 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:45:37.121066 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.121026 2573 generic.go:356] "Generic (PLEG): container finished" podID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerID="5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591" exitCode=0 Jul 07 13:45:37.121490 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.121078 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerDied","Data":"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591"} Jul 07 13:45:37.121490 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.121430 2573 scope.go:117] "RemoveContainer" containerID="5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591" Jul 07 13:45:37.122447 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.122415 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:37.122561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.122498 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:37.122561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.122550 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:45:37.880627 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:37.880592 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:45:38.127372 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:38.127340 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerStarted","Data":"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61"} Jul 07 13:45:38.624282 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:38.624242 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:38.625558 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:38.625531 2573 patch_prober.go:28] interesting pod/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 07 13:45:38.625754 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:38.625730 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 07 13:45:39.066564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:39.066474 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:39.624671 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:39.624631 2573 patch_prober.go:28] interesting pod/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 07 13:45:39.625306 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:39.625272 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 07 13:45:40.625305 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:40.625259 2573 patch_prober.go:28] interesting pod/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 07 13:45:40.625797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:40.625328 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 07 13:45:41.624777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:41.624727 2573 patch_prober.go:28] interesting pod/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 07 13:45:41.625110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:41.624816 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 07 13:45:42.627930 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:42.627892 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:45:42.629101 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:45:42.629078 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:46:05.293269 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:46:05.293233 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:46:18.583374 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:46:18.583290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:46:58.679325 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:46:58.679288 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:47:02.065467 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:02.065433 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:47:02.065827 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:02.065671 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" containerID="cri-o://c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61" gracePeriod=30 Jul 07 13:47:07.301610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.301585 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:47:07.361025 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.360901 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361025 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.360960 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361025 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361002 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361025 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361086 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rbn9\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361117 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361140 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361173 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361329 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361197 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo\") pod \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\" (UID: \"e0d3e46d-bfa9-445d-b42c-1742fd62e15b\") " Jul 07 13:47:07.361624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361359 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:07.361624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361370 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:07.361624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361486 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.361624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361505 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.361624 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361507 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:07.361855 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361681 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:47:07.361855 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.361842 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:07.363342 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.363322 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:47:07.363459 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.363441 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9" (OuterVolumeSpecName: "kube-api-access-5rbn9") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "kube-api-access-5rbn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:07.363532 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.363512 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:47:07.363577 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.363520 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0d3e46d-bfa9-445d-b42c-1742fd62e15b" (UID: "e0d3e46d-bfa9-445d-b42c-1742fd62e15b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:47:07.462583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462554 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462577 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462586 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462595 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462604 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462611 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rbn9\" (UniqueName: \"kubernetes.io/projected/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-kube-api-access-5rbn9\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.462762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.462620 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0d3e46d-bfa9-445d-b42c-1742fd62e15b-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:47:07.492364 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.492334 2573 generic.go:356] "Generic (PLEG): container finished" podID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerID="c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61" exitCode=0 Jul 07 13:47:07.492482 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.492386 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" Jul 07 13:47:07.492482 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.492400 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerDied","Data":"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61"} Jul 07 13:47:07.492482 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.492449 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn" event={"ID":"e0d3e46d-bfa9-445d-b42c-1742fd62e15b","Type":"ContainerDied","Data":"7fec97ee734e994130edefadc900e6df970267d4d44fea5163c6ad90c2b1e668"} Jul 07 13:47:07.492482 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.492480 2573 scope.go:117] "RemoveContainer" containerID="c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61" Jul 07 13:47:07.500892 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.500875 2573 scope.go:117] "RemoveContainer" containerID="5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591" Jul 07 13:47:07.508129 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.508112 2573 scope.go:117] "RemoveContainer" containerID="c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61" Jul 07 13:47:07.508356 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:47:07.508339 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61\": container with ID starting with c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61 not found: ID does not exist" containerID="c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61" Jul 07 13:47:07.508396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.508365 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61"} err="failed to get container status \"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61\": rpc error: code = NotFound desc = could not find container \"c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61\": container with ID starting with c84fda2166d3dc81d0833185a88d58272f08096b7c5a618deed6f07dab5f5f61 not found: ID does not exist" Jul 07 13:47:07.508396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.508381 2573 scope.go:117] "RemoveContainer" containerID="5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591" Jul 07 13:47:07.508603 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:47:07.508589 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591\": container with ID starting with 5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591 not found: ID does not exist" containerID="5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591" Jul 07 13:47:07.508639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.508606 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591"} err="failed to get container status \"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591\": rpc error: code = NotFound desc = could not find container \"5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591\": container with ID starting with 5b9ba2629a523671180882ac7c4f0bbd797c4489a2de4f98c29d3ad22361f591 not found: ID does not exist" Jul 07 13:47:07.513270 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.513247 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:47:07.519257 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.519237 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e1603ba5-openshift-default-6bdbf995bc-ch6pn"] Jul 07 13:47:07.683855 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:07.683830 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:47:08.431230 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:08.431196 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" path="/var/lib/kubelet/pods/e0d3e46d-bfa9-445d-b42c-1742fd62e15b/volumes" Jul 07 13:47:18.883905 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:18.883874 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:47:56.703929 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.703820 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:47:56.704476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704441 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.704476 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704460 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.704581 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704486 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.704581 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704495 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.704675 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704604 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.704675 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.704618 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d3e46d-bfa9-445d-b42c-1742fd62e15b" containerName="istio-proxy" Jul 07 13:47:56.708111 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.708087 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.710766 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.710748 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-55216d-openshift-default-dockercfg-4xzz5\"" Jul 07 13:47:56.717362 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.717341 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:47:56.853495 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853495 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853543 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m82j\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853704 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.853938 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.853716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955170 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955170 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955124 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955221 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955281 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m82j\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955739 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955681 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955739 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955709 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.955843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955780 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.956001 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.955961 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.956121 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.956097 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.957469 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.957440 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.957713 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.957692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.962950 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.962904 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m82j\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:56.963258 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:56.963238 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token\") pod \"e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:57.019368 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.019339 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:57.148088 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.148056 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:47:57.150657 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:47:57.150621 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5360e25d_6f88_42f0_8e58_8bd94b9dee53.slice/crio-874b1f0c04a78822f14ddc3d18443db88e5e89a1d2ecf42214c868859b0dbf99 WatchSource:0}: Error finding container 874b1f0c04a78822f14ddc3d18443db88e5e89a1d2ecf42214c868859b0dbf99: Status 404 returned error can't find the container with id 874b1f0c04a78822f14ddc3d18443db88e5e89a1d2ecf42214c868859b0dbf99 Jul 07 13:47:57.152670 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.152641 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:47:57.152745 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.152708 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:47:57.152745 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.152739 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:47:57.694726 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.694689 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerStarted","Data":"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9"} Jul 07 13:47:57.694952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.694755 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerStarted","Data":"874b1f0c04a78822f14ddc3d18443db88e5e89a1d2ecf42214c868859b0dbf99"} Jul 07 13:47:57.715202 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:57.715158 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podStartSLOduration=1.715141185 podStartE2EDuration="1.715141185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:47:57.71386877 +0000 UTC m=+1891.778766722" watchObservedRunningTime="2026-07-07 13:47:57.715141185 +0000 UTC m=+1891.780039117" Jul 07 13:47:58.020539 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:58.020459 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:58.025439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:58.025414 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:58.699617 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:58.699591 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:47:58.700604 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:47:58.700582 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:48:04.856130 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:04.856096 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:48:04.859267 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:04.859247 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:04.862516 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:04.862497 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-efd33e-openshift-default-dockercfg-v7fxt\"" Jul 07 13:48:04.872408 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:04.872382 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:48:05.025842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.025842 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025843 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025865 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025880 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccbx2\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.025974 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.026047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026076 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.026074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.026303 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.026117 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.126946 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.126903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.126992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127045 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127242 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127242 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccbx2\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127339 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127338 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127651 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127633 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127755 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127735 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.127846 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.127829 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.129295 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.129280 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.129611 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.129594 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.134601 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.134564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.135356 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.135333 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccbx2\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2\") pod \"e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.170252 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.170224 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:05.303326 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.303301 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:48:05.305891 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:48:05.305865 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c3fb37_5e5a_47e9_a2f0_24666a738c1c.slice/crio-73a08971cba51459ea9320fbf5bcc0c5b08c84f239fd181306345107c89312bd WatchSource:0}: Error finding container 73a08971cba51459ea9320fbf5bcc0c5b08c84f239fd181306345107c89312bd: Status 404 returned error can't find the container with id 73a08971cba51459ea9320fbf5bcc0c5b08c84f239fd181306345107c89312bd Jul 07 13:48:05.308057 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.308024 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:05.308148 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.308105 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:05.308208 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.308149 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:05.729096 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.729054 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerStarted","Data":"1a2ada3f10aca45627dc4f80e0292239782077646cd0cf044bea42d53f9491ef"} Jul 07 13:48:05.729269 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.729117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerStarted","Data":"73a08971cba51459ea9320fbf5bcc0c5b08c84f239fd181306345107c89312bd"} Jul 07 13:48:05.747149 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:05.747100 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podStartSLOduration=1.747084596 podStartE2EDuration="1.747084596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:48:05.746297426 +0000 UTC m=+1899.811195396" watchObservedRunningTime="2026-07-07 13:48:05.747084596 +0000 UTC m=+1899.811982545" Jul 07 13:48:06.171034 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:06.171004 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:06.175761 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:06.175740 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:06.732983 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:06.732950 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:06.733899 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:06.733882 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:07.284831 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:07.284796 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:48:14.741145 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:14.741102 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 13:48:14.741771 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:14.741735 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 13:48:15.772444 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.772413 2573 generic.go:356] "Generic (PLEG): container finished" podID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerID="c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9" exitCode=0 Jul 07 13:48:15.772870 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.772493 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerDied","Data":"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9"} Jul 07 13:48:15.772941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.772873 2573 scope.go:117] "RemoveContainer" containerID="c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9" Jul 07 13:48:15.773936 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.773895 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:15.774005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.773982 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:15.774041 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:15.774017 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:16.779063 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:16.779028 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerStarted","Data":"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520"} Jul 07 13:48:17.020237 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:17.020196 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:48:17.020439 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:17.020418 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 07 13:48:17.020482 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:17.020464 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 07 13:48:17.285698 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:17.285665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:48:18.020505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:18.020474 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 07 13:48:18.020885 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:18.020521 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 07 13:48:19.020400 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:19.020365 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 07 13:48:19.020608 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:19.020424 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 07 13:48:20.020245 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:20.020207 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 07 13:48:20.020414 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:20.020276 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 07 13:48:21.020347 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.020310 2573 patch_prober.go:28] interesting pod/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 07 13:48:21.020722 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.020371 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 07 13:48:21.734374 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.734292 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 07 13:48:21.734374 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.734342 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 07 13:48:21.801593 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.801562 2573 generic.go:356] "Generic (PLEG): container finished" podID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerID="1a2ada3f10aca45627dc4f80e0292239782077646cd0cf044bea42d53f9491ef" exitCode=0 Jul 07 13:48:21.801749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.801634 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerDied","Data":"1a2ada3f10aca45627dc4f80e0292239782077646cd0cf044bea42d53f9491ef"} Jul 07 13:48:21.802066 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.802051 2573 scope.go:117] "RemoveContainer" containerID="1a2ada3f10aca45627dc4f80e0292239782077646cd0cf044bea42d53f9491ef" Jul 07 13:48:21.803022 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.802995 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:21.803103 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.803077 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:21.803138 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:21.803130 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:48:22.023495 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:22.023411 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:48:22.023968 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:22.023676 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:48:22.024471 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:22.024450 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:48:22.807447 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:22.807409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerStarted","Data":"6cc2e90817e7d10e8f5cbb033e7a39f7e4134dcfe0085e8a18e0e36ec7603fed"} Jul 07 13:48:23.170705 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:23.170662 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:23.171109 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:23.170952 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 07 13:48:23.171109 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:23.170998 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 07 13:48:24.171551 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:24.171514 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 07 13:48:24.171953 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:24.171616 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 07 13:48:25.171340 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:25.171298 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 07 13:48:25.171519 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:25.171356 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 07 13:48:26.171264 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:26.171225 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 07 13:48:26.171645 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:26.171281 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 07 13:48:27.174532 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:27.174505 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:27.174952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:27.174835 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:27.175528 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:27.175511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:48:28.380098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:28.380062 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:48:39.187982 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:48:39.187942 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:49:13.479124 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:49:13.479088 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:49:36.580535 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:49:36.580496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:50:15.239473 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:15.239439 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:50:15.240057 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:15.239667 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" containerID="cri-o://a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520" gracePeriod=30 Jul 07 13:50:18.083219 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:18.083184 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:50:20.487044 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.487014 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:50:20.579964 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.579864 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.579964 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.579904 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m82j\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.579964 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.579959 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.579975 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.579993 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580043 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580084 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580115 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580149 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy\") pod \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\" (UID: \"5360e25d-6f88-42f0-8e58-8bd94b9dee53\") " Jul 07 13:50:20.580221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580182 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:20.580564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580243 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:20.580564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580425 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.580564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580445 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.580564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580524 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:20.580740 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.580569 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:50:20.581216 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.581192 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data" (OuterVolumeSpecName: "istio-data") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:20.582378 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.582352 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:20.582496 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.582437 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:50:20.582613 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.582596 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token" (OuterVolumeSpecName: "istio-token") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:20.582719 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.582703 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j" (OuterVolumeSpecName: "kube-api-access-5m82j") pod "5360e25d-6f88-42f0-8e58-8bd94b9dee53" (UID: "5360e25d-6f88-42f0-8e58-8bd94b9dee53"). InnerVolumeSpecName "kube-api-access-5m82j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:20.681289 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681253 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m82j\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-kube-api-access-5m82j\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681289 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681283 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681289 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681294 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681302 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681310 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681317 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:20.681520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:20.681324 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5360e25d-6f88-42f0-8e58-8bd94b9dee53-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:21.297633 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.297597 2573 generic.go:356] "Generic (PLEG): container finished" podID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerID="a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520" exitCode=0 Jul 07 13:50:21.297799 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.297642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerDied","Data":"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520"} Jul 07 13:50:21.297799 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.297661 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" Jul 07 13:50:21.297799 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.297718 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx" event={"ID":"5360e25d-6f88-42f0-8e58-8bd94b9dee53","Type":"ContainerDied","Data":"874b1f0c04a78822f14ddc3d18443db88e5e89a1d2ecf42214c868859b0dbf99"} Jul 07 13:50:21.297799 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.297743 2573 scope.go:117] "RemoveContainer" containerID="a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520" Jul 07 13:50:21.306703 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.306684 2573 scope.go:117] "RemoveContainer" containerID="c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9" Jul 07 13:50:21.314030 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.314013 2573 scope.go:117] "RemoveContainer" containerID="a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520" Jul 07 13:50:21.314272 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:50:21.314253 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520\": container with ID starting with a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520 not found: ID does not exist" containerID="a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520" Jul 07 13:50:21.314316 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.314281 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520"} err="failed to get container status \"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520\": rpc error: code = NotFound desc = could not find container \"a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520\": container with ID starting with a8fc218acf94d842413bb30cc029e5744d09d1b28a68f28b2a44515205e13520 not found: ID does not exist" Jul 07 13:50:21.314316 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.314298 2573 scope.go:117] "RemoveContainer" containerID="c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9" Jul 07 13:50:21.314498 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:50:21.314481 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9\": container with ID starting with c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9 not found: ID does not exist" containerID="c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9" Jul 07 13:50:21.314536 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.314503 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9"} err="failed to get container status \"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9\": rpc error: code = NotFound desc = could not find container \"c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9\": container with ID starting with c23bf79ba7101f6f1183ae07e33879506ce8fd712bc18e26dea896cac514c4b9 not found: ID does not exist" Jul 07 13:50:21.318241 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.318218 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:50:21.321705 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:21.321685 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-55216d-openshift-default-776c865f8c-t8szx"] Jul 07 13:50:22.170729 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:22.170650 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:50:22.171199 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:22.170883 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" containerID="cri-o://6cc2e90817e7d10e8f5cbb033e7a39f7e4134dcfe0085e8a18e0e36ec7603fed" gracePeriod=30 Jul 07 13:50:22.432699 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:22.432622 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" path="/var/lib/kubelet/pods/5360e25d-6f88-42f0-8e58-8bd94b9dee53/volumes" Jul 07 13:50:23.099834 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.099783 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:50:23.100110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100096 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.100167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100112 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.100167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100124 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.100167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100129 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.100270 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100192 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.100270 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.100201 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5360e25d-6f88-42f0-8e58-8bd94b9dee53" containerName="istio-proxy" Jul 07 13:50:23.105010 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.104988 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.107479 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.107458 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6839ee-openshift-default-dockercfg-l6mxn\"" Jul 07 13:50:23.113702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.113679 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:50:23.205320 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb4q6\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.205777 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.205619 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306021 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.305992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb4q6\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306137 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306211 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306523 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306523 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306523 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306475 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306523 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306495 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306599 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.306775 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.306756 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.307102 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.307083 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.308887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.308863 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.309123 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.309105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.318828 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.318803 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.319230 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.319212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb4q6\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6\") pod \"e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.415735 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.415691 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:23.551927 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.551891 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:50:23.554140 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:50:23.554113 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a61c6e2_5f1a_4ffb_85fb_9755295c2a98.slice/crio-3365b6672e22337c48ae5cedaa0fc42a97f3f5ba0b3913b0f061fb716ae77017 WatchSource:0}: Error finding container 3365b6672e22337c48ae5cedaa0fc42a97f3f5ba0b3913b0f061fb716ae77017: Status 404 returned error can't find the container with id 3365b6672e22337c48ae5cedaa0fc42a97f3f5ba0b3913b0f061fb716ae77017 Jul 07 13:50:23.556042 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.556013 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:23.556110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.556077 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:23.556110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:23.556104 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:24.313281 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:24.313238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerStarted","Data":"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862"} Jul 07 13:50:24.313650 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:24.313301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerStarted","Data":"3365b6672e22337c48ae5cedaa0fc42a97f3f5ba0b3913b0f061fb716ae77017"} Jul 07 13:50:24.340538 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:24.340476 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podStartSLOduration=1.3404564350000001 podStartE2EDuration="1.340456435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:50:24.339691627 +0000 UTC m=+2038.404589598" watchObservedRunningTime="2026-07-07 13:50:24.340456435 +0000 UTC m=+2038.405354388" Jul 07 13:50:24.416125 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:24.416092 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:24.421107 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:24.421085 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:25.318176 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:25.318144 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:25.319033 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:25.319013 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:27.179307 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.179274 2573 patch_prober.go:28] interesting pod/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": read tcp 10.133.0.2:46824->10.133.0.27:15021: read: connection reset by peer" start-of-body= Jul 07 13:50:27.179830 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.179335 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": read tcp 10.133.0.2:46824->10.133.0.27:15021: read: connection reset by peer" Jul 07 13:50:27.327792 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.327760 2573 generic.go:356] "Generic (PLEG): container finished" podID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerID="6cc2e90817e7d10e8f5cbb033e7a39f7e4134dcfe0085e8a18e0e36ec7603fed" exitCode=0 Jul 07 13:50:27.327997 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.327832 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerDied","Data":"6cc2e90817e7d10e8f5cbb033e7a39f7e4134dcfe0085e8a18e0e36ec7603fed"} Jul 07 13:50:27.327997 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.327905 2573 scope.go:117] "RemoveContainer" containerID="1a2ada3f10aca45627dc4f80e0292239782077646cd0cf044bea42d53f9491ef" Jul 07 13:50:27.417986 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.417967 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:50:27.544452 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544372 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccbx2\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544452 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544415 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544452 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544431 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544560 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544609 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544654 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544672 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544701 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544700 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:27.544721 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544713 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:27.545086 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.544732 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert\") pod \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\" (UID: \"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c\") " Jul 07 13:50:27.545086 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.545032 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.545086 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.545054 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.545240 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.545120 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:50:27.545421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.545393 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:27.545507 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.545453 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:27.546824 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.546789 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2" (OuterVolumeSpecName: "kube-api-access-ccbx2") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "kube-api-access-ccbx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:27.546979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.546828 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:27.546979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.546861 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:27.546979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.546872 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" (UID: "e9c3fb37-5e5a-47e9-a2f0-24666a738c1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:50:27.645843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645806 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.645843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645835 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.645843 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645848 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccbx2\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-kube-api-access-ccbx2\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.646098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645860 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.646098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645871 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.646098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645882 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:27.646098 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:27.645893 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:28.333336 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.333310 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" Jul 07 13:50:28.333793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.333300 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w" event={"ID":"e9c3fb37-5e5a-47e9-a2f0-24666a738c1c","Type":"ContainerDied","Data":"73a08971cba51459ea9320fbf5bcc0c5b08c84f239fd181306345107c89312bd"} Jul 07 13:50:28.333793 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.333458 2573 scope.go:117] "RemoveContainer" containerID="6cc2e90817e7d10e8f5cbb033e7a39f7e4134dcfe0085e8a18e0e36ec7603fed" Jul 07 13:50:28.354973 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.354947 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:50:28.361583 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.361560 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-efd33e-openshift-default-58647dfdff-zvs8w"] Jul 07 13:50:28.432200 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:28.432175 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" path="/var/lib/kubelet/pods/e9c3fb37-5e5a-47e9-a2f0-24666a738c1c/volumes" Jul 07 13:50:30.682326 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:30.682286 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:50:31.592939 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.592890 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:50:31.593181 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593168 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.593228 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593182 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.593228 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593196 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.593228 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593201 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.593321 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593255 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.593321 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.593264 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c3fb37-5e5a-47e9-a2f0-24666a738c1c" containerName="istio-proxy" Jul 07 13:50:31.596480 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.596460 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.599389 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.599328 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1904bd-openshift-default-dockercfg-mk24v\"" Jul 07 13:50:31.609223 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.609201 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:50:31.784573 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784615 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784639 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkflf\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784673 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784729 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784783 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784805 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.785073 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.784831 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885615 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885789 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885789 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885641 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885789 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkflf\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885970 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885970 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885835 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885970 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.885970 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.885906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886180 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886241 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886215 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886326 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886383 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886353 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886443 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886421 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.886576 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.886558 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.888053 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.888032 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.888265 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.888248 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.894114 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.894088 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.894254 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.894233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkflf\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf\") pod \"e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:31.906995 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:31.906974 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:32.082806 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.082771 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:50:32.085705 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:50:32.085678 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2971c3_830e_4562_9ef3_a41823bf19fc.slice/crio-d386aa047b074ced497cfdd13dddb0f495abb62b024f3605131e1a47aa2d8f2a WatchSource:0}: Error finding container d386aa047b074ced497cfdd13dddb0f495abb62b024f3605131e1a47aa2d8f2a: Status 404 returned error can't find the container with id d386aa047b074ced497cfdd13dddb0f495abb62b024f3605131e1a47aa2d8f2a Jul 07 13:50:32.087702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.087672 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:32.087757 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.087741 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:32.087792 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.087771 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:32.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.354471 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerStarted","Data":"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b"} Jul 07 13:50:32.354688 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.354527 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerStarted","Data":"d386aa047b074ced497cfdd13dddb0f495abb62b024f3605131e1a47aa2d8f2a"} Jul 07 13:50:32.372755 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.372701 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podStartSLOduration=1.372683501 podStartE2EDuration="1.372683501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:50:32.371628345 +0000 UTC m=+2046.436526323" watchObservedRunningTime="2026-07-07 13:50:32.372683501 +0000 UTC m=+2046.437581452" Jul 07 13:50:32.908038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:32.908002 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:33.913215 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:33.913186 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:34.362283 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:34.362249 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:34.363323 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:34.363301 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:39.386226 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.386193 2573 generic.go:356] "Generic (PLEG): container finished" podID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerID="65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862" exitCode=0 Jul 07 13:50:39.386645 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.386269 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerDied","Data":"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862"} Jul 07 13:50:39.386645 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.386616 2573 scope.go:117] "RemoveContainer" containerID="65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862" Jul 07 13:50:39.387527 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.387505 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:39.387578 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.387570 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:39.387622 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.387602 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:39.792779 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:39.792688 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:50:40.319006 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:40.318972 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:40.391863 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:40.391829 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerStarted","Data":"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f"} Jul 07 13:50:40.416730 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:40.416697 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:40.417727 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:40.417705 2573 patch_prober.go:28] interesting pod/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 13:50:40.417782 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:40.417752 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 13:50:41.416460 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:41.416427 2573 patch_prober.go:28] interesting pod/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 13:50:41.416817 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:41.416490 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 13:50:42.416198 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:42.416168 2573 patch_prober.go:28] interesting pod/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 13:50:42.416375 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:42.416218 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 13:50:43.416409 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:43.416370 2573 patch_prober.go:28] interesting pod/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 07 13:50:43.416819 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:43.416432 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 07 13:50:44.420302 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:44.420265 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:44.420680 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:44.420569 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:44.425322 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:44.425293 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:48.427797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.427702 2573 generic.go:356] "Generic (PLEG): container finished" podID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerID="548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b" exitCode=0 Jul 07 13:50:48.431467 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.431436 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerDied","Data":"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b"} Jul 07 13:50:48.431829 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.431811 2573 scope.go:117] "RemoveContainer" containerID="548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b" Jul 07 13:50:48.432779 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.432751 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:48.432865 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.432818 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:48.432865 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:48.432852 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:50:49.362632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:49.362599 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:49.435726 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:49.435683 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerStarted","Data":"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e"} Jul 07 13:50:49.907561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:49.907533 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:49.908747 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:49.908717 2573 patch_prober.go:28] interesting pod/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 13:50:49.908873 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:49.908764 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 13:50:50.387278 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:50.387243 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:50:50.907690 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:50.907660 2573 patch_prober.go:28] interesting pod/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 13:50:50.908070 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:50.907713 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 13:50:51.907706 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:51.907666 2573 patch_prober.go:28] interesting pod/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 13:50:51.908117 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:51.907726 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 13:50:52.908270 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:52.908225 2573 patch_prober.go:28] interesting pod/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 07 13:50:52.908656 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:52.908289 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 07 13:50:53.610836 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:53.610802 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:50:53.611057 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:53.611030 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" containerID="cri-o://191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f" gracePeriod=30 Jul 07 13:50:53.911637 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:53.911610 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:53.912053 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:53.911870 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:53.912575 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:53.912558 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:50:58.865130 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.865102 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:58.924661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924581 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924617 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924636 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb4q6\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924661 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924655 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924699 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924720 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924743 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924781 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924853 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data\") pod \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\" (UID: \"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98\") " Jul 07 13:50:58.924999 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.924872 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:58.925299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.925117 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:58.925299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.925051 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:50:58.925299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.925245 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:58.925460 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.925318 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:58.926128 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.926101 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:58.926901 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.926881 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:50:58.927014 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.926935 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:58.927014 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.926956 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:50:58.927086 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:58.927007 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6" (OuterVolumeSpecName: "kube-api-access-gb4q6") pod "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" (UID: "2a61c6e2-5f1a-4ffb-85fb-9755295c2a98"). InnerVolumeSpecName "kube-api-access-gb4q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:50:59.025837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025809 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.025837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025834 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.025837 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025843 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb4q6\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-kube-api-access-gb4q6\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.026056 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025852 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.026056 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025861 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.026056 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025868 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.026056 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025876 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.026056 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.025883 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:50:59.487244 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.487214 2573 generic.go:356] "Generic (PLEG): container finished" podID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerID="191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f" exitCode=0 Jul 07 13:50:59.487429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.487259 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" Jul 07 13:50:59.487429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.487297 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerDied","Data":"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f"} Jul 07 13:50:59.487429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.487354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc" event={"ID":"2a61c6e2-5f1a-4ffb-85fb-9755295c2a98","Type":"ContainerDied","Data":"3365b6672e22337c48ae5cedaa0fc42a97f3f5ba0b3913b0f061fb716ae77017"} Jul 07 13:50:59.487429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.487380 2573 scope.go:117] "RemoveContainer" containerID="191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f" Jul 07 13:50:59.496249 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.496226 2573 scope.go:117] "RemoveContainer" containerID="65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862" Jul 07 13:50:59.503429 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.503412 2573 scope.go:117] "RemoveContainer" containerID="191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f" Jul 07 13:50:59.503661 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:50:59.503643 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f\": container with ID starting with 191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f not found: ID does not exist" containerID="191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f" Jul 07 13:50:59.503705 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.503670 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f"} err="failed to get container status \"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f\": rpc error: code = NotFound desc = could not find container \"191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f\": container with ID starting with 191d446e49ba53190b40106ba40880c1a3b16df38866a64ad5c4c362409ce91f not found: ID does not exist" Jul 07 13:50:59.503705 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.503692 2573 scope.go:117] "RemoveContainer" containerID="65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862" Jul 07 13:50:59.503904 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:50:59.503890 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862\": container with ID starting with 65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862 not found: ID does not exist" containerID="65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862" Jul 07 13:50:59.503961 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.503909 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862"} err="failed to get container status \"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862\": rpc error: code = NotFound desc = could not find container \"65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862\": container with ID starting with 65d7a25e878e84249e57603e23ae325ff87734624c69aafef0ac31c3ae3c9862 not found: ID does not exist" Jul 07 13:50:59.508196 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.508171 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:50:59.512028 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:50:59.512009 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6839ee-openshift-default-69f48769dc-chbfc"] Jul 07 13:51:00.418409 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:00.418380 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:51:00.418868 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:00.418621 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" containerID="cri-o://c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e" gracePeriod=30 Jul 07 13:51:00.432472 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:00.432440 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" path="/var/lib/kubelet/pods/2a61c6e2-5f1a-4ffb-85fb-9755295c2a98/volumes" Jul 07 13:51:01.311413 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.311380 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:51:01.311782 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.311768 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:01.311872 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.311819 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:01.311958 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.311941 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:01.311958 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.311957 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:01.314202 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.314181 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.316794 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.316773 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-820d4f-openshift-default-dockercfg-lmt7z\"" Jul 07 13:51:01.326368 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.326350 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:51:01.447037 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447117 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447159 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447210 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447295 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.447421 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.447316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdznk\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548467 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548482 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548507 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdznk\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548536 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.548711 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.548706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.549273 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.549035 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.549273 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.549074 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.549273 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.549125 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.549273 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.549267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.549489 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.549449 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.551005 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.550981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.551160 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.551142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.557077 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.557036 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.557444 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.557414 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdznk\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk\") pod \"e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.624047 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.624018 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:01.753941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.753907 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:51:01.756479 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:51:01.756444 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7e4c6d_3496_4a08_939e_13b12e44cc76.slice/crio-b2b7bec09ea42135d2ac7c72c16941f8dcf869b4335ab049b090376e48cf1c72 WatchSource:0}: Error finding container b2b7bec09ea42135d2ac7c72c16941f8dcf869b4335ab049b090376e48cf1c72: Status 404 returned error can't find the container with id b2b7bec09ea42135d2ac7c72c16941f8dcf869b4335ab049b090376e48cf1c72 Jul 07 13:51:01.758663 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.758629 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:01.758772 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.758707 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:01.758772 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:01.758754 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:02.507408 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.507375 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerStarted","Data":"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892"} Jul 07 13:51:02.507798 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.507425 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerStarted","Data":"b2b7bec09ea42135d2ac7c72c16941f8dcf869b4335ab049b090376e48cf1c72"} Jul 07 13:51:02.529327 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.529277 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podStartSLOduration=1.529262241 podStartE2EDuration="1.529262241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:51:02.526646821 +0000 UTC m=+2076.591544773" watchObservedRunningTime="2026-07-07 13:51:02.529262241 +0000 UTC m=+2076.594160217" Jul 07 13:51:02.624492 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.624455 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:02.629376 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.629353 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:02.882531 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:02.882498 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:51:03.512168 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:03.512139 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:03.513096 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:03.513076 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:05.660838 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.660816 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:51:05.786931 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.786831 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.786931 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.786868 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.786931 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.786901 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.786981 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787021 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkflf\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787071 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787107 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787146 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787194 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787152 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:51:05.787572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787172 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert\") pod \"9e2971c3-830e-4562-9ef3-a41823bf19fc\" (UID: \"9e2971c3-830e-4562-9ef3-a41823bf19fc\") " Jul 07 13:51:05.787572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787318 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:51:05.787572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787485 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.787572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787495 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:51:05.787572 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787505 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.787804 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.787783 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:51:05.788037 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.788012 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:51:05.789200 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.789172 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf" (OuterVolumeSpecName: "kube-api-access-lkflf") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "kube-api-access-lkflf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:51:05.789297 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.789223 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:51:05.789579 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.789564 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:51:05.789629 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.789572 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e2971c3-830e-4562-9ef3-a41823bf19fc" (UID: "9e2971c3-830e-4562-9ef3-a41823bf19fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:51:05.888753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888711 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkflf\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-kube-api-access-lkflf\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.888753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888750 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.888753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888759 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.888753 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888768 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e2971c3-830e-4562-9ef3-a41823bf19fc-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.889023 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888776 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.889023 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888785 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:05.889023 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:05.888793 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e2971c3-830e-4562-9ef3-a41823bf19fc-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:51:06.526478 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.526445 2573 generic.go:356] "Generic (PLEG): container finished" podID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerID="c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e" exitCode=0 Jul 07 13:51:06.526640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.526504 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" Jul 07 13:51:06.526640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.526530 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerDied","Data":"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e"} Jul 07 13:51:06.526640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.526614 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq" event={"ID":"9e2971c3-830e-4562-9ef3-a41823bf19fc","Type":"ContainerDied","Data":"d386aa047b074ced497cfdd13dddb0f495abb62b024f3605131e1a47aa2d8f2a"} Jul 07 13:51:06.526640 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.526639 2573 scope.go:117] "RemoveContainer" containerID="c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e" Jul 07 13:51:06.535075 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.534791 2573 scope.go:117] "RemoveContainer" containerID="548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b" Jul 07 13:51:06.541945 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.541905 2573 scope.go:117] "RemoveContainer" containerID="c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e" Jul 07 13:51:06.542214 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:51:06.542196 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e\": container with ID starting with c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e not found: ID does not exist" containerID="c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e" Jul 07 13:51:06.542279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.542223 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e"} err="failed to get container status \"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e\": rpc error: code = NotFound desc = could not find container \"c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e\": container with ID starting with c2ff222177ab0d6d9c9a21014905a9342f7da3a276f93093434444483ab3f04e not found: ID does not exist" Jul 07 13:51:06.542279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.542245 2573 scope.go:117] "RemoveContainer" containerID="548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b" Jul 07 13:51:06.542584 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:51:06.542545 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b\": container with ID starting with 548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b not found: ID does not exist" containerID="548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b" Jul 07 13:51:06.542739 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.542579 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b"} err="failed to get container status \"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b\": rpc error: code = NotFound desc = could not find container \"548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b\": container with ID starting with 548208c29b01e372dc86df9ba33dc74f1cea4e03d3e8883d8d96c04272674f1b not found: ID does not exist" Jul 07 13:51:06.544610 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.544590 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:51:06.548795 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:06.548776 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1904bd-openshift-default-84b7f75b77-t8fgq"] Jul 07 13:51:08.432122 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:08.432090 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" path="/var/lib/kubelet/pods/9e2971c3-830e-4562-9ef3-a41823bf19fc/volumes" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621437 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621834 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621849 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621871 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621880 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a61c6e2-5f1a-4ffb-85fb-9755295c2a98" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621893 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.621902 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.622035 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.624941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.622047 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2971c3-830e-4562-9ef3-a41823bf19fc" containerName="istio-proxy" Jul 07 13:51:09.630613 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.630584 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.638689 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.638656 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:51:09.641437 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.640028 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c9e14a-openshift-default-dockercfg-6pbhm\"" Jul 07 13:51:09.725416 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725375 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87s9x\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725623 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725623 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725623 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725640 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725755 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.725801 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.726039 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.725805 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.826562 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.826523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.826836 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.826818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.827130 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827111 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.827269 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.827420 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.827523 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827509 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.827632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827619 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827699 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827722 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827750 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.827767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s9x\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.828006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.828345 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.828038 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.829797 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.829750 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.837971 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.831799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.841491 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.841459 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.841708 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.841679 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s9x\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x\") pod \"e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:09.958689 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:09.958599 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:10.109401 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.109371 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:51:10.113351 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:51:10.113320 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfceab3b4_9231_4d2b_af6d_62c07b2d3653.slice/crio-c756666a696498f6b55e7ea9cc74f3e43b9f9369dd276734d6b0f6b87cebfec5 WatchSource:0}: Error finding container c756666a696498f6b55e7ea9cc74f3e43b9f9369dd276734d6b0f6b87cebfec5: Status 404 returned error can't find the container with id c756666a696498f6b55e7ea9cc74f3e43b9f9369dd276734d6b0f6b87cebfec5 Jul 07 13:51:10.115432 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.115403 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:10.115518 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.115479 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:10.115518 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.115508 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:10.555189 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.555150 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerStarted","Data":"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8"} Jul 07 13:51:10.555518 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.555202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerStarted","Data":"c756666a696498f6b55e7ea9cc74f3e43b9f9369dd276734d6b0f6b87cebfec5"} Jul 07 13:51:10.575983 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.575933 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podStartSLOduration=1.5759020129999999 podStartE2EDuration="1.575902013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:51:10.574257848 +0000 UTC m=+2084.639155823" watchObservedRunningTime="2026-07-07 13:51:10.575902013 +0000 UTC m=+2084.640799942" Jul 07 13:51:10.959497 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:10.959446 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:11.964525 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:11.964498 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:12.563606 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:12.563566 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:12.564629 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:12.564605 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:13.581213 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:13.581121 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:51:16.585611 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.585581 2573 generic.go:356] "Generic (PLEG): container finished" podID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerID="bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892" exitCode=0 Jul 07 13:51:16.586035 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.585627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerDied","Data":"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892"} Jul 07 13:51:16.586084 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.586033 2573 scope.go:117] "RemoveContainer" containerID="bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892" Jul 07 13:51:16.587029 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.586999 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:16.587107 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.587090 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:16.587158 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:16.587142 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:17.592047 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:17.592006 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerStarted","Data":"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871"} Jul 07 13:51:17.624803 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:17.624764 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:17.625877 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:17.625842 2573 patch_prober.go:28] interesting pod/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 13:51:17.626040 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:17.625936 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 13:51:18.512909 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:18.512876 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:18.625233 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:18.625179 2573 patch_prober.go:28] interesting pod/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 13:51:18.625728 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:18.625280 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 13:51:19.624691 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:19.624649 2573 patch_prober.go:28] interesting pod/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 13:51:19.624896 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:19.624711 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 13:51:20.624897 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:20.624850 2573 patch_prober.go:28] interesting pod/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 07 13:51:20.625401 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:20.624962 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 07 13:51:21.638768 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:21.638725 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:21.643221 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:21.643192 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:51:24.288514 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:24.288479 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:51:27.565367 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.565293 2573 patch_prober.go:28] interesting pod/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 13:51:27.565745 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.565358 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 13:51:27.634040 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.634006 2573 generic.go:356] "Generic (PLEG): container finished" podID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerID="bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8" exitCode=0 Jul 07 13:51:27.634190 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.634086 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerDied","Data":"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8"} Jul 07 13:51:27.634521 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.634506 2573 scope.go:117] "RemoveContainer" containerID="bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8" Jul 07 13:51:27.635564 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.635532 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:27.635632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.635615 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:27.635681 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:27.635666 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 07 13:51:28.643164 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:28.643122 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerStarted","Data":"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944"} Jul 07 13:51:28.959888 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:28.959787 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:28.960110 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:28.960085 2573 patch_prober.go:28] interesting pod/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 13:51:28.960168 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:28.960150 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 13:51:29.959206 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:29.959169 2573 patch_prober.go:28] interesting pod/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 13:51:29.959580 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:29.959229 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 13:51:30.959517 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:30.959480 2573 patch_prober.go:28] interesting pod/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 13:51:30.959941 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:30.959548 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 13:51:31.959314 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:31.959281 2573 patch_prober.go:28] interesting pod/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 07 13:51:31.959497 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:31.959353 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 07 13:51:32.962952 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:32.962907 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:32.963396 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:32.963255 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:32.963869 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:32.963854 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:51:34.891898 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:34.891864 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:51:57.183071 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:51:57.183036 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:52:13.286885 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:52:13.286849 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:52:30.982616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:52:30.982540 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:52:44.578477 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:52:44.578441 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:52:59.580760 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:52:59.580728 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:53:17.290644 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:17.290612 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:53:33.390543 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:33.390509 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:53:46.987591 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:46.987559 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:53:58.497580 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.497500 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:53:58.500052 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.500035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.502835 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.502810 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-820d4f\"/\"openshift-service-ca.crt\"" Jul 07 13:53:58.503791 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.503769 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-820d4f\"/\"default-dockercfg-b9gpc\"" Jul 07 13:53:58.503895 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.503770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-820d4f\"/\"sub-test-model-820d4f-kserve-self-signed-certs\"" Jul 07 13:53:58.503895 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.503861 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-820d4f\"/\"kube-root-ca.crt\"" Jul 07 13:53:58.510117 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.510098 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:53:58.551652 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.551766 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7ljr\" (UniqueName: \"kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.551766 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551704 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.551766 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551755 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.551879 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551783 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.551879 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.551801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652671 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652702 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7ljr\" (UniqueName: \"kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652974 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652737 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652974 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652974 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.652974 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.652805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.653231 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.653213 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.653289 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.653256 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.653334 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.653285 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.655162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.655138 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.655261 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.655232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.660362 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.660345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7ljr\" (UniqueName: \"kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr\") pod \"sub-test-model-820d4f-kserve-77987cb895-t7x24\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.811689 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.811597 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:53:58.937547 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:58.937411 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:53:58.941585 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:53:58.941557 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa664b02_d75a_4e44_a430_e70510b6d837.slice/crio-453d89f2ccca18c1f9fced51d5670a9ca15e51a1e73dfd62b53f42d254307d73 WatchSource:0}: Error finding container 453d89f2ccca18c1f9fced51d5670a9ca15e51a1e73dfd62b53f42d254307d73: Status 404 returned error can't find the container with id 453d89f2ccca18c1f9fced51d5670a9ca15e51a1e73dfd62b53f42d254307d73 Jul 07 13:53:59.254932 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:59.254882 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" event={"ID":"fa664b02-d75a-4e44-a430-e70510b6d837","Type":"ContainerStarted","Data":"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c"} Jul 07 13:53:59.254932 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:59.254935 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" event={"ID":"fa664b02-d75a-4e44-a430-e70510b6d837","Type":"ContainerStarted","Data":"453d89f2ccca18c1f9fced51d5670a9ca15e51a1e73dfd62b53f42d254307d73"} Jul 07 13:53:59.271179 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:53:59.271136 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" podStartSLOduration=1.271122587 podStartE2EDuration="1.271122587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 13:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:53:59.270135432 +0000 UTC m=+2253.335033380" watchObservedRunningTime="2026-07-07 13:53:59.271122587 +0000 UTC m=+2253.336020539" Jul 07 13:54:02.092779 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:02.092748 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:54:08.812703 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:08.812657 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:54:08.812703 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:08.812714 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:54:08.824882 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:08.824861 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:54:09.308514 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:09.308481 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:54:22.883414 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:22.883382 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:54:37.785465 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:37.785422 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:54:53.882371 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:53.882340 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:54:57.083737 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:54:57.083703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:55:08.902611 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:08.902562 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" podUID="fa664b02-d75a-4e44-a430-e70510b6d837" containerName="main" containerID="cri-o://42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c" gracePeriod=45 Jul 07 13:55:09.144043 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.144019 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:55:09.248887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248805 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7ljr\" (UniqueName: \"kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.248887 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248864 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.249118 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248902 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.249118 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248952 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.249118 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248972 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.249118 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.248995 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home\") pod \"fa664b02-d75a-4e44-a430-e70510b6d837\" (UID: \"fa664b02-d75a-4e44-a430-e70510b6d837\") " Jul 07 13:55:09.249331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.249200 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:09.249331 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.249262 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:09.249404 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.249332 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home" (OuterVolumeSpecName: "home") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:09.251145 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.251124 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 13:55:09.251232 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.251169 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr" (OuterVolumeSpecName: "kube-api-access-s7ljr") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "kube-api-access-s7ljr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:55:09.251232 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.251207 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm" (OuterVolumeSpecName: "dshm") pod "fa664b02-d75a-4e44-a430-e70510b6d837" (UID: "fa664b02-d75a-4e44-a430-e70510b6d837"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:55:09.350104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350057 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-tmp-dir\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.350104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350096 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-dshm\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.350104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350107 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-home\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.350104 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350116 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7ljr\" (UniqueName: \"kubernetes.io/projected/fa664b02-d75a-4e44-a430-e70510b6d837-kube-api-access-s7ljr\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.350359 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350125 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa664b02-d75a-4e44-a430-e70510b6d837-tls-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.350359 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.350134 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa664b02-d75a-4e44-a430-e70510b6d837-model-cache\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:55:09.556250 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.556160 2573 generic.go:356] "Generic (PLEG): container finished" podID="fa664b02-d75a-4e44-a430-e70510b6d837" containerID="42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c" exitCode=0 Jul 07 13:55:09.556250 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.556199 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" Jul 07 13:55:09.556250 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.556227 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" event={"ID":"fa664b02-d75a-4e44-a430-e70510b6d837","Type":"ContainerDied","Data":"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c"} Jul 07 13:55:09.556492 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.556257 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24" event={"ID":"fa664b02-d75a-4e44-a430-e70510b6d837","Type":"ContainerDied","Data":"453d89f2ccca18c1f9fced51d5670a9ca15e51a1e73dfd62b53f42d254307d73"} Jul 07 13:55:09.556492 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.556273 2573 scope.go:117] "RemoveContainer" containerID="42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c" Jul 07 13:55:09.565220 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.565202 2573 scope.go:117] "RemoveContainer" containerID="42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c" Jul 07 13:55:09.565506 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:55:09.565488 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c\": container with ID starting with 42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c not found: ID does not exist" containerID="42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c" Jul 07 13:55:09.565557 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.565514 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c"} err="failed to get container status \"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c\": rpc error: code = NotFound desc = could not find container \"42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c\": container with ID starting with 42bccb253616d7a0a7fe397ff34e5cd2cc5d86e19315a2c2b227510ee2ad150c not found: ID does not exist" Jul 07 13:55:09.576621 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.576596 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:55:09.580561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:09.580539 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-820d4f/sub-test-model-820d4f-kserve-77987cb895-t7x24"] Jul 07 13:55:10.432255 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:10.432224 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa664b02-d75a-4e44-a430-e70510b6d837" path="/var/lib/kubelet/pods/fa664b02-d75a-4e44-a430-e70510b6d837/volumes" Jul 07 13:55:12.679784 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:12.679749 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:55:27.778516 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:27.778486 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:55:56.890651 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:55:56.890618 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:56:10.986561 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:56:10.986525 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:56:24.186328 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:56:24.186293 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:57:02.009886 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:02.009850 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:57:02.010894 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:02.010148 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" containerID="cri-o://baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871" gracePeriod=30 Jul 07 13:57:04.784944 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:04.784886 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sfnsw"] Jul 07 13:57:06.641182 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:06.641152 2573 patch_prober.go:28] interesting pod/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 13:57:06.641664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:06.641219 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 13:57:07.256616 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.256594 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:57:07.353744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353662 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.353744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353702 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdznk\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.353744 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353727 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353761 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353805 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353834 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353864 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353948 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353973 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy\") pod \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\" (UID: \"7e7e4c6d-3496-4a08-939e-13b12e44cc76\") " Jul 07 13:57:07.354127 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.353983 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:07.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354143 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:07.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354222 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:07.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354246 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354261 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.354505 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354423 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:57:07.354765 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.354691 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:07.356144 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.356117 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:57:07.356255 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.356153 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:57:07.356541 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.356522 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:07.356597 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.356541 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk" (OuterVolumeSpecName: "kube-api-access-kdznk") pod "7e7e4c6d-3496-4a08-939e-13b12e44cc76" (UID: "7e7e4c6d-3496-4a08-939e-13b12e44cc76"). InnerVolumeSpecName "kube-api-access-kdznk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:57:07.454627 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454597 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454627 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454620 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454627 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454628 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdznk\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-kube-api-access-kdznk\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454638 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454646 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454654 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7e4c6d-3496-4a08-939e-13b12e44cc76-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:07.454810 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:07.454661 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7e4c6d-3496-4a08-939e-13b12e44cc76-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:08.040039 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.040006 2573 generic.go:356] "Generic (PLEG): container finished" podID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerID="baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871" exitCode=0 Jul 07 13:57:08.040520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.040065 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" Jul 07 13:57:08.040520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.040099 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerDied","Data":"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871"} Jul 07 13:57:08.040520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.040150 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84" event={"ID":"7e7e4c6d-3496-4a08-939e-13b12e44cc76","Type":"ContainerDied","Data":"b2b7bec09ea42135d2ac7c72c16941f8dcf869b4335ab049b090376e48cf1c72"} Jul 07 13:57:08.040520 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.040173 2573 scope.go:117] "RemoveContainer" containerID="baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871" Jul 07 13:57:08.053468 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.053448 2573 scope.go:117] "RemoveContainer" containerID="bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892" Jul 07 13:57:08.060950 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.060896 2573 scope.go:117] "RemoveContainer" containerID="baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871" Jul 07 13:57:08.061178 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:57:08.061161 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871\": container with ID starting with baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871 not found: ID does not exist" containerID="baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871" Jul 07 13:57:08.061232 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.061185 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871"} err="failed to get container status \"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871\": rpc error: code = NotFound desc = could not find container \"baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871\": container with ID starting with baa6113b2cfe4b90380bc93c36070fe894713c1e6454345389c13c63bb8cc871 not found: ID does not exist" Jul 07 13:57:08.061232 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.061200 2573 scope.go:117] "RemoveContainer" containerID="bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892" Jul 07 13:57:08.061426 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:57:08.061403 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892\": container with ID starting with bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892 not found: ID does not exist" containerID="bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892" Jul 07 13:57:08.061464 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.061435 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892"} err="failed to get container status \"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892\": rpc error: code = NotFound desc = could not find container \"bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892\": container with ID starting with bd924569b5a325f0c4fc676078fa4513d80cfb14d6e5ec983f7699890a4fb892 not found: ID does not exist" Jul 07 13:57:08.064294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.064273 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:57:08.070633 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.070603 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-820d4f-openshift-default-fdd76b67f-tmf84"] Jul 07 13:57:08.431809 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.431779 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" path="/var/lib/kubelet/pods/7e7e4c6d-3496-4a08-939e-13b12e44cc76/volumes" Jul 07 13:57:08.957294 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.957264 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:57:08.957508 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:08.957482 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" containerID="cri-o://98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944" gracePeriod=30 Jul 07 13:57:14.199749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.199724 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:57:14.304490 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304413 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87s9x\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304490 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304442 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304490 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304463 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304500 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304525 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304620 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304675 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304699 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.304749 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304741 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token\") pod \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\" (UID: \"fceab3b4-9231-4d2b-af6d-62c07b2d3653\") " Jul 07 13:57:14.305090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304776 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:14.305090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304757 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:14.305090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304889 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 13:57:14.305090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.304994 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:14.305299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.305114 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-credential-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.305299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.305135 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istiod-ca-cert\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.305299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.305149 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-socket\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.305299 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.305161 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-workload-certs\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.305579 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.305547 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data" (OuterVolumeSpecName: "istio-data") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:14.306762 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.306734 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x" (OuterVolumeSpecName: "kube-api-access-87s9x") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "kube-api-access-87s9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:57:14.306884 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.306835 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token" (OuterVolumeSpecName: "istio-token") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 13:57:14.307182 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.307156 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 13:57:14.307182 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.307154 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fceab3b4-9231-4d2b-af6d-62c07b2d3653" (UID: "fceab3b4-9231-4d2b-af6d-62c07b2d3653"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 13:57:14.406019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.405993 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-envoy\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.406019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.406013 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-token\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.406019 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.406023 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87s9x\" (UniqueName: \"kubernetes.io/projected/fceab3b4-9231-4d2b-af6d-62c07b2d3653-kube-api-access-87s9x\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.406236 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.406032 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-data\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.406236 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.406041 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fceab3b4-9231-4d2b-af6d-62c07b2d3653-istio-podinfo\") on node \"ip-10-0-133-20.ec2.internal\" DevicePath \"\"" Jul 07 13:57:14.947428 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:14.947399 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-ww4sd_ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7/cleanup/0.log" Jul 07 13:57:15.071385 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.071353 2573 generic.go:356] "Generic (PLEG): container finished" podID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerID="98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944" exitCode=0 Jul 07 13:57:15.071555 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.071397 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" Jul 07 13:57:15.071555 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.071437 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerDied","Data":"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944"} Jul 07 13:57:15.071555 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.071485 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5" event={"ID":"fceab3b4-9231-4d2b-af6d-62c07b2d3653","Type":"ContainerDied","Data":"c756666a696498f6b55e7ea9cc74f3e43b9f9369dd276734d6b0f6b87cebfec5"} Jul 07 13:57:15.071555 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.071508 2573 scope.go:117] "RemoveContainer" containerID="98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944" Jul 07 13:57:15.079645 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.079459 2573 scope.go:117] "RemoveContainer" containerID="bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8" Jul 07 13:57:15.086819 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.086804 2573 scope.go:117] "RemoveContainer" containerID="98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944" Jul 07 13:57:15.087107 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:57:15.087065 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944\": container with ID starting with 98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944 not found: ID does not exist" containerID="98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944" Jul 07 13:57:15.087229 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.087118 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944"} err="failed to get container status \"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944\": rpc error: code = NotFound desc = could not find container \"98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944\": container with ID starting with 98c6eb97075c5e3d5ef66257d214eff5784ad08a2a8320b7565484b925ad7944 not found: ID does not exist" Jul 07 13:57:15.087229 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.087150 2573 scope.go:117] "RemoveContainer" containerID="bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8" Jul 07 13:57:15.087531 ip-10-0-133-20 kubenswrapper[2573]: E0707 13:57:15.087505 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8\": container with ID starting with bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8 not found: ID does not exist" containerID="bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8" Jul 07 13:57:15.087785 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.087548 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8"} err="failed to get container status \"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8\": rpc error: code = NotFound desc = could not find container \"bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8\": container with ID starting with bc17b0cccd8437e18dda6f4564294404a19827b3d2381c33f8fe0342daff81a8 not found: ID does not exist" Jul 07 13:57:15.088733 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.088716 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:57:15.092038 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.092018 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c9e14a-openshift-default-5cc7f4d679-zz8l5"] Jul 07 13:57:15.830671 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.830645 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fbc88fbc-t7k4v_b26ba531-5055-44df-a4a0-b935043ae01c/copy-manifests/0.log" Jul 07 13:57:15.853351 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:15.853329 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fbc88fbc-t7k4v_b26ba531-5055-44df-a4a0-b935043ae01c/manager/0.log" Jul 07 13:57:16.432631 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:16.432598 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" path="/var/lib/kubelet/pods/fceab3b4-9231-4d2b-af6d-62c07b2d3653/volumes" Jul 07 13:57:18.105291 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:18.105264 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sfnsw_834bf5a8-f347-4ed5-8831-dab1e5ea8bca/limitador/0.log" Jul 07 13:57:18.597681 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:18.597648 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h9kdk_ec299975-b4bc-4557-b54a-618346342c5f/istio-proxy/0.log" Jul 07 13:57:18.726835 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:18.726801 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-58rqr_cffd934a-f74a-4ddb-ac3b-8daeba401117/discovery/0.log" Jul 07 13:57:20.189743 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:20.189712 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbh4bv4_c038e44f-154f-4a68-977f-f6d0bbcaa1c8/main/0.log" Jul 07 13:57:27.862945 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:27.862901 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qplbh_7ec2ac7c-f972-4867-a382-81944fe7f93d/global-pull-secret-syncer/0.log" Jul 07 13:57:27.983880 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:27.983850 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d4zs8_0ebc4978-2fc4-4c32-b612-960ffd5bd966/konnectivity-agent/0.log" Jul 07 13:57:28.049598 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:28.049573 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-20.ec2.internal_b69ec39edf2c7e27054c97d97fb57365/haproxy/0.log" Jul 07 13:57:33.676815 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:33.676783 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sfnsw_834bf5a8-f347-4ed5-8831-dab1e5ea8bca/limitador/0.log" Jul 07 13:57:35.849903 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:35.849868 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94nsf_7a43f530-62f7-4a51-9084-4c8e1f364796/node-exporter/0.log" Jul 07 13:57:35.872111 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:35.872084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94nsf_7a43f530-62f7-4a51-9084-4c8e1f364796/kube-rbac-proxy/0.log" Jul 07 13:57:35.896786 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:35.896767 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-94nsf_7a43f530-62f7-4a51-9084-4c8e1f364796/init-textfile/0.log" Jul 07 13:57:38.206141 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206112 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq"] Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206372 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206382 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206392 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206397 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206406 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206412 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206425 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa664b02-d75a-4e44-a430-e70510b6d837" containerName="main" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206430 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa664b02-d75a-4e44-a430-e70510b6d837" containerName="main" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206474 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa664b02-d75a-4e44-a430-e70510b6d837" containerName="main" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206483 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e7e4c6d-3496-4a08-939e-13b12e44cc76" containerName="istio-proxy" Jul 07 13:57:38.206589 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.206489 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fceab3b4-9231-4d2b-af6d-62c07b2d3653" containerName="istio-proxy" Jul 07 13:57:38.209141 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.209125 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.211956 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.211923 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4dq5n\"/\"openshift-service-ca.crt\"" Jul 07 13:57:38.212099 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.211995 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4dq5n\"/\"kube-root-ca.crt\"" Jul 07 13:57:38.212162 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.212128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4dq5n\"/\"default-dockercfg-5mq4r\"" Jul 07 13:57:38.218639 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.218616 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq"] Jul 07 13:57:38.304682 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.304655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-lib-modules\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.304682 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.304684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-sys\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.304904 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.304706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-podres\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.304904 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.304723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfz4s\" (UniqueName: \"kubernetes.io/projected/c45ec397-c6bd-435e-8725-4613e149aad9-kube-api-access-mfz4s\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.304904 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.304827 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-proc\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.405977 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.405943 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-podres\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406047 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.405988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfz4s\" (UniqueName: \"kubernetes.io/projected/c45ec397-c6bd-435e-8725-4613e149aad9-kube-api-access-mfz4s\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406047 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406042 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-proc\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406138 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406109 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-lib-modules\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406138 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-podres\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406138 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-sys\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406187 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-sys\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406216 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-proc\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.406279 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.406268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c45ec397-c6bd-435e-8725-4613e149aad9-lib-modules\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.414351 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.414330 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfz4s\" (UniqueName: \"kubernetes.io/projected/c45ec397-c6bd-435e-8725-4613e149aad9-kube-api-access-mfz4s\") pod \"perf-node-gather-daemonset-44rdq\" (UID: \"c45ec397-c6bd-435e-8725-4613e149aad9\") " pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.519738 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.519651 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:38.646676 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:38.646653 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq"] Jul 07 13:57:38.649129 ip-10-0-133-20 kubenswrapper[2573]: W0707 13:57:38.649104 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc45ec397_c6bd_435e_8725_4613e149aad9.slice/crio-059f1662f4671ff2c5b9ca28d20fa28251554821739a9e37582bf6849d39b585 WatchSource:0}: Error finding container 059f1662f4671ff2c5b9ca28d20fa28251554821739a9e37582bf6849d39b585: Status 404 returned error can't find the container with id 059f1662f4671ff2c5b9ca28d20fa28251554821739a9e37582bf6849d39b585 Jul 07 13:57:39.164090 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:39.164059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" event={"ID":"c45ec397-c6bd-435e-8725-4613e149aad9","Type":"ContainerStarted","Data":"6654c6e641d74d215490fc01b08a2a1c0ad3db183281599684171bc45e2b3bd5"} Jul 07 13:57:39.164272 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:39.164110 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" event={"ID":"c45ec397-c6bd-435e-8725-4613e149aad9","Type":"ContainerStarted","Data":"059f1662f4671ff2c5b9ca28d20fa28251554821739a9e37582bf6849d39b585"} Jul 07 13:57:39.164272 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:39.164131 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:39.180632 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:39.180584 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" podStartSLOduration=1.18056893 podStartE2EDuration="1.18056893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 13:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 13:57:39.178660832 +0000 UTC m=+2473.243558785" watchObservedRunningTime="2026-07-07 13:57:39.18056893 +0000 UTC m=+2473.245466881" Jul 07 13:57:40.474246 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:40.474212 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwsnb_d9d6646f-baa5-4e91-96c0-e614c0f43800/dns/0.log" Jul 07 13:57:40.496265 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:40.496231 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cwsnb_d9d6646f-baa5-4e91-96c0-e614c0f43800/kube-rbac-proxy/0.log" Jul 07 13:57:40.600664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:40.600635 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fnngb_c313195a-53b5-4a03-a686-d4740f5b112d/dns-node-resolver/0.log" Jul 07 13:57:41.189680 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:41.189635 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zllhn_6977b697-1c8a-4994-a3e0-1430604adb23/node-ca/0.log" Jul 07 13:57:42.090535 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:42.090502 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h9kdk_ec299975-b4bc-4557-b54a-618346342c5f/istio-proxy/0.log" Jul 07 13:57:42.278653 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:42.278625 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-58rqr_cffd934a-f74a-4ddb-ac3b-8daeba401117/discovery/0.log" Jul 07 13:57:43.152979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:43.152951 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6jmp_a1a98509-a7f6-475b-8a44-9f12c809a00c/serve-healthcheck-canary/0.log" Jul 07 13:57:43.679880 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:43.679852 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q4rl_b409881a-909e-4fbc-a79e-e448a6eaee5d/kube-rbac-proxy/0.log" Jul 07 13:57:43.698613 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:43.698589 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q4rl_b409881a-909e-4fbc-a79e-e448a6eaee5d/exporter/0.log" Jul 07 13:57:43.717621 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:43.717587 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2q4rl_b409881a-909e-4fbc-a79e-e448a6eaee5d/extractor/0.log" Jul 07 13:57:45.177326 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:45.177301 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4dq5n/perf-node-gather-daemonset-44rdq" Jul 07 13:57:46.081673 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:46.081642 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723850-ww4sd_ebd5a256-9b9d-4a89-beca-5c8ea6f1edc7/cleanup/0.log" Jul 07 13:57:46.484517 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:46.484482 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fbc88fbc-t7k4v_b26ba531-5055-44df-a4a0-b935043ae01c/manager/0.log" Jul 07 13:57:46.503586 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:46.503564 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69fbc88fbc-t7k4v_b26ba531-5055-44df-a4a0-b935043ae01c/copy-manifests/0.log" Jul 07 13:57:48.077979 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:48.077948 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76d5cc9db8-p5m4s_1742be8f-79c5-4b71-aa00-2ab6f7ab55ab/manager/0.log" Jul 07 13:57:48.657870 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:48.657842 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h9kdk_ec299975-b4bc-4557-b54a-618346342c5f/istio-proxy/0.log" Jul 07 13:57:48.845137 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:48.845092 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-58rqr_cffd934a-f74a-4ddb-ac3b-8daeba401117/discovery/0.log" Jul 07 13:57:50.204204 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:50.204172 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h9kdk_ec299975-b4bc-4557-b54a-618346342c5f/istio-proxy/0.log" Jul 07 13:57:50.392937 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:50.392889 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-58rqr_cffd934a-f74a-4ddb-ac3b-8daeba401117/discovery/0.log" Jul 07 13:57:53.106664 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:53.106631 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sd8fr_7577368b-a09b-4d8c-b7fa-e3e52feaed6c/migrator/0.log" Jul 07 13:57:53.124747 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:53.124716 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sd8fr_7577368b-a09b-4d8c-b7fa-e3e52feaed6c/graceful-termination/0.log" Jul 07 13:57:54.492167 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.492141 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5949d_5e2e72de-b475-4cd9-86da-8a38b8567446/kube-multus/0.log" Jul 07 13:57:54.684224 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.684195 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/kube-multus-additional-cni-plugins/0.log" Jul 07 13:57:54.706363 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.706340 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/egress-router-binary-copy/0.log" Jul 07 13:57:54.726364 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.726335 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/cni-plugins/0.log" Jul 07 13:57:54.746742 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.746683 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/bond-cni-plugin/0.log" Jul 07 13:57:54.765710 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.765688 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/routeoverride-cni/0.log" Jul 07 13:57:54.787631 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.787610 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/whereabouts-cni-bincopy/0.log" Jul 07 13:57:54.809097 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:54.809078 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qtd6q_f038fc69-abff-4174-bfdf-01d488dd6efd/whereabouts-cni/0.log" Jul 07 13:57:55.005047 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:55.004973 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-28m8c_6f5f807f-62cd-4965-a59e-952b2e12356c/network-metrics-daemon/0.log" Jul 07 13:57:55.023542 ip-10-0-133-20 kubenswrapper[2573]: I0707 13:57:55.023522 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-28m8c_6f5f807f-62cd-4965-a59e-952b2e12356c/kube-rbac-proxy/0.log"